Selected Research
My research interests focus on efficient algorithms and systems for deep learning, specifically
large foundation models.
|
|
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Guangxuan Xiao,
Jiaming Tang,
Jingwei Zuo,
Junxian Guo,
Shang Yang,
Haotian Tang,
Yao Fu,
Song Han
arXiv
[paper]
[code]
[demo]
|
|
Efficient Streaming Language Models with Attention Sinks
Guangxuan Xiao,
Yuandong Tian,
Beidi Chen,
Song Han,
Mike Lewis
ICLR 2024
[paper]
[code]
[MIT
News]
[NVIDIA
TensorRT-LLM]
[on iPhone]
|
|
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao*,
Ji Lin*,
Mickael Seznec,
Hao Wu,
Julien Demouth,
Song Han
ICML 2023
[paper]
[code]
[NVIDIA
TensorRT-LLM]
|
|
FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
Guangxuan Xiao*,
Tianwei Yin*,
William T. Freeman,
Frédo Durand,
Song Han
IJCV 2024
[website]
[paper]
[code]
|
|
Massachusetts Institute of Technology
2022.08 - Present
Ph.D student in Computer Science
Advisor: Prof. Song Han
|
|
Tsinghua University
2018.08 - 2022.07
B.Eng. in Computer Science
B.Econ. in Economics (Second Major)
Advisor: Prof. Zhiyuan Liu.
|
|
Stanford University
2020.07 - 2021.06
Visiting Research Student
Advisor: Prof. Jure Leskovec
Mentor: Jiaxuan You
|
|
Stanford University
2021.06 - 2021.11
Visiting Research Student through the UGVR
program
Advisor: Prof. Jiajun Wu, Prof. Leslie Pack Kaelbling
Mentor: Jiayuan Mao
|
Honors & Awards
- Hewlett Packard Fellowship, 2022
- Boeing Scholarship, 2021
- Tsinghua "Future Scholar" Scientific Research Grant (30000$), 2021
- National Scholarship, 2020
- Contemporary Undergraduate Mathematical Contest in Modeling, 1st Prize, 2020
- Beijing "Challenge Cup" Academic Science and Technology Competition, 1st Prize, 2020
- Tsinghua Comprehensive Excellence Scholarship, 2019
|
Miscellaneous
I love to play soccer. I was the captain and striker of my department soccer team.
I also love to play table tennis, Go (Weiqi), and piano. Beethoven's works are my favorite.
|
|