Zhongkai Yu

I am currently a second-year PhD student in the CSE department of UC San Diego (since 2024), working with Prof.Yufei Ding. Before coming to UCSD, I obtained my master’s degree from Institute of Computing Technology, Chinese Academy of Sciences, under the supervision of Prof. Yunji Chen. Prior to that, I received my bachelor’s degree from Shanghai Jiao Tong University.

My research centers on the co-evolution of LLM and hardware systems: I explore novel architectures and systems to accelerate LLM workloads, while harnessing LLMs to advance chip design. My work lies at the intersection of Computer Architecture, ML Systems, and AI4Chip.

Also, shout-out to my roommate Zaifeng, who is an expert in kernels, systems, and “cooking”. Check out his homepage and research.

Email: zhy055@ucsd.edu

Education

University of California, San Diego (UCSD), 2024 - Present
Ph.D. student in Computer Science & Engineering
University of Chinese Academy of Sciences (UCAS), 2021 - 2024
M.E. in Computer Technology
Shanghai Jiao Tong University (SJTU), 2017 - 2021
B.S. in Physics (Zhiyuan Honors Program)

Experience

Samsung Semiconductor, 2025.6 - 2025.9.
Research Intern, AGI Lab
Cambricon Technologies, 2022.3 - 2024.3.
IC Design Intern, Architecture group

Selected Publications

ArXiv

ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design

Zhongkai Yu, Chenyang Zhou, Yichen Lin, Hejia Zhang, Haotian Ye, Junxia Cui, Zaifeng Pan, Jishen Zhao, and Yufei Ding

arXiv preprint arXiv:2601.21448, 2026

Bib PDF

@article{yu2026chipbenchnextstepbenchmarkevaluating,
  title = {ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design},
  author = {Yu, Zhongkai and Zhou, Chenyang and Lin, Yichen and Zhang, Hejia and Ye, Haotian and Cui, Junxia and Pan, Zaifeng and Zhao, Jishen and Ding, Yufei},
  journal = {arXiv preprint arXiv:2601.21448},
  year = {2026},
}

ArXiv

AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving

Zhongkai Yu, Haotian Ye, Chenyang Zhou, Ohm Rishabh Venkatachalam, Zaifeng Pan, Zhengding Hu, Junsung Kim, Won Woo Ro, Po-An Tsai, Shuyi Pei, Yangwook Kang, and Yufei Ding

arXiv preprint arXiv:2604.26103, 2026

Bib PDF

@article{yu2026ammamultichipletmemorycentricarchitecture,
  title = {AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving},
  author = {Yu, Zhongkai and Ye, Haotian and Zhou, Chenyang and Venkatachalam, Ohm Rishabh and Pan, Zaifeng and Hu, Zhengding and Kim, Junsung and Ro, Won Woo and Tsai, Po-An and Pei, Shuyi and Kang, Yangwook and Ding, Yufei},
  journal = {arXiv preprint arXiv:2604.26103},
  year = {2026},
}

ISCA’26

Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference

Zhongkai Yu, Yue Guan, Zihao Yu, Chenyang Zhou, Zhengding Hu, Shuyi Pei, Yangwook Kang, Yufei Ding, and Po-An Tsai

arXiv preprint arXiv:2510.05497, 2026

Best Paper Candidate

Bib PDF

@article{yu2026patternschaosforecastingdata,
  title = {Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference},
  author = {Yu, Zhongkai and Guan, Yue and Yu, Zihao and Zhou, Chenyang and Hu, Zhengding and Pei, Shuyi and Kang, Yangwook and Ding, Yufei and Tsai, Po-An},
  journal = {arXiv preprint arXiv:2510.05497},
  year = {2026},
}

MICRO’24

Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM

Zhongkai Yu, Shengwen Liang, Tianyun Ma, Yunke Cai, Ziyuan Nan, Di Huang, Xinkai Song, Yifan Hao, Jie Zhang, Tian Zhi, Yongwei Zhao, Zidong Du, Xing Hu, Qi Guo, and Tianshi Chen

In Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture, 2024

Bib PDF

@inproceedings{yu2024cambricon,
  title = {Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM},
  author = {Yu, Zhongkai and Liang, Shengwen and Ma, Tianyun and Cai, Yunke and Nan, Ziyuan and Huang, Di and Song, Xinkai and Hao, Yifan and Zhang, Jie and Zhi, Tian and Zhao, Yongwei and Du, Zidong and Hu, Xing and Guo, Qi and Chen, Tianshi},
  booktitle = {Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture},
  pages = {1474--1488},
  year = {2024},
  organization = {IEEE},
}

DAC’22

E2sr: an end-to-end video codec assisted system for super resolution acceleration

Zhuoran Song, Zhongkai Yu, Naifeng Jing, and Xiaoyao Liang

In Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022

Bib PDF

@inproceedings{song2022e2sr,
  title = {E2sr: an end-to-end video codec assisted system for super resolution acceleration},
  author = {Song, Zhuoran and Yu, Zhongkai and Jing, Naifeng and Liang, Xiaoyao},
  pages = {229--234},
  booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
  year = {2022},
}