Guangxuan Xiao   肖光烜

I am a third-year Ph.D. candidate at MIT EECS, advised by Prof. Song Han. I graduated from the Department of Computer Science and Technology at Tsinghua University in 2022, advised by Prof. Zhiyuan Liu. I have been very fortunate to work with Prof. Jure Leskovec as a summer intern in 2020, with Prof. Jiajun Wu and Prof. Leslie Pack Kaelbling through the Stanford UGVR Program.

Email  /  Google Scholar  /  Github /  Linkedin

profile photo
Selected Research

My research interests focus on efficient algorithms and systems for deep learning, specifically large foundation models.

DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, Song Han
arXiv
[paper] [code] [demo]

Efficient Streaming Language Models with Attention Sinks
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis
ICLR 2024
[paper] [code] [MIT News] [NVIDIA TensorRT-LLM] [on iPhone]

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao*, Ji Lin*, Mickael Seznec, Hao Wu, Julien Demouth, Song Han
ICML 2023
[paper] [code] [NVIDIA TensorRT-LLM]

FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
Guangxuan Xiao*, Tianwei Yin*, William T. Freeman, Frédo Durand, Song Han
IJCV 2024
[website] [paper] [code]

Experience
Massachusetts Institute of Technology
2022.08 - Present

Ph.D student in Computer Science
Advisor: Prof. Song Han
Tsinghua University
2018.08 - 2022.07

B.Eng. in Computer Science
B.Econ. in Economics (Second Major)
Advisor: Prof. Zhiyuan Liu.
Stanford University
2020.07 - 2021.06

Visiting Research Student
Advisor: Prof. Jure Leskovec
Mentor: Jiaxuan You
Stanford University
2021.06 - 2021.11

Visiting Research Student through the UGVR program
Advisor: Prof. Jiajun Wu, Prof. Leslie Pack Kaelbling
Mentor: Jiayuan Mao
Honors & Awards
  • Hewlett Packard Fellowship, 2022
  • Boeing Scholarship, 2021
  • Tsinghua "Future Scholar" Scientific Research Grant (30000$), 2021
  • National Scholarship, 2020
  • Contemporary Undergraduate Mathematical Contest in Modeling, 1st Prize, 2020
  • Beijing "Challenge Cup" Academic Science and Technology Competition, 1st Prize, 2020
  • Tsinghua Comprehensive Excellence Scholarship, 2019
Miscellaneous

I love to play soccer. I was the captain and striker of my department soccer team.
I also love to play table tennis, Go (Weiqi), and piano. Beethoven's works are my favorite.


Template adapted from Jon Barron.