Fan Yang

personal email:

yang DOT fan AT 163 DOT com

work email: fanyang AT microsoft DOT com

My name is Fan Yang (杨凡 in Chinese). I am a systems researcher and research manager of the Systems Research Group (SRG) at Microsoft Research Asia (MSR-Asia). I joined MSR-Asia after receiving my doctoral degree and bachelor’s degree in Computer Science at Nanjing University.

My research passion lies in Computer Systems. My recent focus is on exploring the fundamental principles of the systems for Artificial Intelligence (AI). I am among the first to discover and advocate the now well-known design principles for AI systems, including the hardware-aligned tile abstraction for AI compilers and the relaxed monotonicity for vector stores. Some techniques and solutions derived from these principles have been open-sourced and adopted by Microsoft products like Azure, M365, and Bing, and the corresponding research results have appeared in top systems conferences like OSDI/SOSP. Some open-source projects like OpenPAI or NNI even incubated new businesses. More recently, I have been passionate about the co-design of AI algorithms and systems, which I believe will define the next chapter of AI. In the past, I worked on large-scale systems, such as graph systems. I co-developed GraM, a high-performance graph engine that set a new speed record for trillion-scale graph analytics.

As a researcher, I also engage in public academic services, including serving on the program committees of ASPLOS (2022), EuroSys (2023, 2025, 2026), ChinaSys (19th).

news

Jul 16, 2025	rStar-Math and LIPS picked as "the most groundbreaking AI papers from the first half of 2025" by Turing Post
Aug 26, 2024	Internship Opportunities at SRG
Aug 20, 2024	MSR-Asia StarTrack Scholars Program

latest posts

Jun 17, 2025	New methods boost reasoning in small and large language models - Microsoft Research
Feb 05, 2025	Advances to low-bit quantization enable LLMs on edge devices - Microsoft Research
Jan 09, 2025	Microsoft's new rStar-Math technique upgrades small models to outperform OpenAI's o1-preview at math problems \| VentureBeat

selected publications

ArXiv

TrainVerify: Equivalence-Based Verification for Distributed LLM Training

Yunchi Lu, and 6 more authors

ArXiv. (A revised version will appear in SOSP’25) , 2025

Bib HTML

@article{trainverify25,
  title = {TrainVerify: Equivalence-Based Verification for Distributed LLM Training},
  author = {Lu, Yunchi and Miao, Youshan and Tan, Cheng and Huang, Peng and Zhu, Yi and Zhang, Xian and Yang, Fan},
  year = {2025},
  journal = {ArXiv},
}

OSDI

WaferLLM: A Wafer-Scale LLM Inference System

Congjie He, and 7 more authors

In 19th USENIX Symposium on Operating Systems Design and Implementation, OSDI, 2025

Bib HTML

@inproceedings{waferllm25,
  title = {WaferLLM: A Wafer-Scale LLM Inference System},
  author = {He, Congjie and Huang, Yeqi and Mu, Pei and Miao, Ziming and Xue, Jilong and Ma, Lingxiao and Yang, Fan and Mai, Luo},
  year = {2025},
  booktitle = {19th {USENIX} Symposium on Operating Systems Design and Implementation, {OSDI}},
}

ArXiv

RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference

Yaoqi Chen, and 17 more authors

ArXiv, 2025

Bib HTML

@article{chen2025retroinfervectorstorageapproachscalable,
  title = {RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference},
  author = {Chen, Yaoqi and Zhang, Jinkai and Lu, Baotong and Zhang, Qianxi and Zhang, Chengruidong and Luo, Jingjia and Liu, Di and Jiang, Huiqiang and Chen, Qi and Liu, Jing and Ding, Bailu and Yan, Xiao and Jiang, Jiawei and Chen, Chen and Zhang, Mingxing and Yang, Yuqing and Yang, Fan and Yang, Mao},
  year = {2025},
  journal = {ArXiv},
}

OSDI

VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity

Qianxi Zhang, and 11 more authors

In 17th USENIX Symposium on Operating Systems Design and Implementation, OSDI, 2023

Bib HTML Code

@inproceedings{DBLP:conf/osdi/ZhangXCSXCCH00Y23,
  title = {{VBASE:} Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity},
  author = {Zhang, Qianxi and Xu, Shuotao and Chen, Qi and Sui, Guoxin and Xie, Jiadong and Cai, Zhizhen and Chen, Yaoqi and He, Yinxuan and Yang, Yuqing and Yang, Fan and Yang, Mao and Zhou, Lidong},
  year = {2023},
  booktitle = {17th {USENIX} Symposium on Operating Systems Design and Implementation, {OSDI}},
}

OSDI

ROLLER: Fast and Efficient Tensor Compilation for Deep Learning

Hongyu Zhu, and 14 more authors

In 16th USENIX Symposium on Operating Systems Design and Implementation, OSDI, 2022

Bib HTML

@inproceedings{DBLP:conf/osdi/ZhuWDKLZXMXC0YZ22,
  title = {{ROLLER:} Fast and Efficient Tensor Compilation for Deep Learning},
  author = {Zhu, Hongyu and Wu, Ruofan and Diao, Yijia and Ke, Shanbin and Li, Haoyu and Zhang, Chen and Xue, Jilong and Ma, Lingxiao and Xia, Yuqing and Cui, Wei and Yang, Fan and Yang, Mao and Zhou, Lidong and Cidon, Asaf and Pekhimenko, Gennady},
  year = {2022},
  booktitle = {16th {USENIX} Symposium on Operating Systems Design and Implementation, {OSDI}},
}