Lijie (Derrick) Yang

I am a senior Computer Science student at Carnegie Mellon University, with double concentrations in computer systems and machine learning. Currently, I’m a research assistant at the Catalyst Lab, where I’m fortunate to work under the guidance of Prof. Zhihao Jia and Prof. Tianqi Chen.

Before this, I conducted research with Prof. Ye-Qiong Song at LORIA lab on real-time networks, and with Prof. Umut Acar on developing language model serving systems. Additionally, I’ve contributed to open-source project at MIT’s HAN Lab, focusing on on-device LLM inference optimization.

Research Interests: My work lies at the intersection of builing efficient deep learning systems with algorithmic innovations. I’m particularly passionate about accelerating large language models, optimizing on-device AI, and enhancing network reliability for real-time applications.

news

Jan 20, 2025	TidalDecode is accepted to ICLR 2025, see you in Singapore !
Nov 14, 2024	Gave a talk at CMU Catalyst Lab on TidalDecode
Oct 08, 2024	Our project on position-persistent sparse attention (PPSA), TidalDecode, is on ArXiv!
Aug 24, 2024	Honored to be an early inductee into Phi Beta Kappa (ΦΒΚ) of Class 2025 !
Jul 03, 2024	BWE is accepted to LCN 2024 !
May 01, 2024	RalmSpec is accepted to ICML 2024 !
Mar 03, 2024	SpecInfer is accepted to ASPLOS 2024 !

selected publications

ICLR

TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention

Lijie Yang^*, Zhihao Zhang^*, Zhuofu Chen, Zikun Li, and Zhihao Jia

to appear at International Conference on Learning Representations, 2025

Bib Code Website

@article{yang2024tidaldecodefastaccuratellm,
  title = {TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention},
  author = {Yang, Lijie and Zhang, Zhihao and Chen, Zhuofu and Li, Zikun and Jia, Zhihao},
  year = {2025},
  eprint = {2410.05076},
  journal = {to appear at International Conference on Learning Representations},
  primaryclass = {cs.LG},
  url = {https://arxiv.org/abs/2410.05076},
}

LCN

Blocking-Waived Estimation: Improving the Worst-Case End-To-End Delay Analysis in Switched Ethernet

Lijie Yang, Théo Docquier, Ludovic Thomas, and Ye-Qiong Song

In Proceedings of Local Computer Networks, 2024

Bib Code Website

@article{lcn2024bwe,
  title = {Blocking-Waived Estimation: Improving the Worst-Case End-To-End Delay Analysis in Switched Ethernet},
  author = {Yang, Lijie and Docquier, Théo and Thomas, Ludovic and Song, Ye-Qiong},
  journal = {In Proceedings of Local Computer Networks},
  year = {2024},
}

ICML

Accelerating Retrieval-augmented Language Model Serving with Speculation

Zhihao Zhang, Alan Zhu, Lijie Yang, Yihua Xu, Lanting Li, Phitchaya Mangpo Phothilimthana, and Zhihao Jia

In Proceedings of International Conference on Machine Learning, 2024

Bib Code Website

@article{zhang2024accelerating,
  title = {Accelerating Retrieval-augmented Language Model Serving with Speculation},
  author = {Zhang, Zhihao and Zhu, Alan and Yang, Lijie and Xu, Yihua and Li, Lanting and Phothilimthana, Phitchaya Mangpo and Jia, Zhihao},
  journal = {In Proceedings of International Conference on Machine Learning},
  year = {2024},
}

ASPLOS

SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification

Xupeng Miao^*, Gabriele Oliaro^*, Zhihao Zhang^*, Xinhao Cheng^*, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, and 6 more authors

In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, Apr 2024

Bib Code Website

@inproceedings{Miao_2024,
  series = {ASPLOS ’24},
  title = {SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification},
  url = {http://dx.doi.org/10.1145/3620666.3651335},
  doi = {10.1145/3620666.3651335},
  booktitle = {Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3},
  publisher = {ACM},
  author = {Miao, Xupeng and Oliaro, Gabriele and Zhang, Zhihao and Cheng, Xinhao and Wang, Zeyu and Zhang, Zhengxin and Wong, Rae Ying Yee and Zhu, Alan and Yang, Lijie and Shi, Xiaoxiang and Shi, Chunan and Chen, Zhuoming and Arfeen, Daiyaan and Abhyankar, Reyna and Jia, Zhihao},
  year = {2024},
  month = apr,
  collection = {ASPLOS ’24},
}