Luo Mai

Associate Professor

University of Edinburgh

About Me

I am a Reader (Associate Professor) in the School of Informatics at the University of Edinburgh, where I lead the Large-Scale Machine Learning Systems Group. I also co-lead the UK EPSRC Centre for Doctoral Training in Machine Learning Systems and a UK ARIA Project on Scaling AI Compute.

My research group works across the full stack of AI systems – from models and data to software and infrastructure – pursuing co-designs that aim to achieve a 1000× leap in efficiency, scalability, and reliability. My work has been recognized with several awards and rising-star honors from both academia and industry. I am passionate about open-source and knowledge exchange. I co-founded open-source AI system libraries that have collectively received over 20,000 GitHub stars and co-edited the open-source textbook Machine Learning Systems: Design and Implementation.

Before joining Edinburgh, I was a Research Associate at Imperial College London, working with Peter Pietzuch, and a Visiting Researcher at Microsoft Research. My PhD, supervised by Paolo Costa and Alexander L. Wolf, was supported by a Google Fellowship in Cloud Computing.

Interests

Computer Systems
Machine Learning
Data Management

Education

PhD in Computer Science, 2018

Imperial College London, UK
MRes in Advanced Computing, 2012

Imperial College London, UK

News

[09/25, Paper] MoE-CAP accepted to NeurIPS 2025 (Dataset and Benchmark Track).

[08/25, Grant] Win a prestigious award to build systems for powering AI4Math breakthroughs.

[07/25, Paper] WaferLLM, the world fastest LLM inference system, accepted to OSDI 2025.

[03/25, Achievement] Starting August 2025, I’ll be Reader (Associate Professor).

[10/24, Grant] Secured a prestigious ARIA grant with Imperial College & Cambridge University.

[10/24, Paper] Tenplex, the first elastic LLM system, accepted to SOSP 2024.

[07/24, Student Achievement] Congrats to Yao Fu on Winning 2024 Rising Star in ML & Systems.

[07/24, Paper] ServerlessLLM, the first serverless LLM system, accepted to OSDI 2024.

[05/24, Award] Win a Microsoft Research Startrack Scholar Award.

[03/24, Grant] Secured funds from EPSRC and industry partners to build a CDT for ML Systems.

See all posts

Publications

Yinsicheng Jiang *, Yao Fu *, Yeqi Huang *, Ping Nie, Zhan Lu, Leyang Xue, Congjie He, Man-Kit Sit, Jilong Xue, Li Dong, Ziming Miao, Dayou Du, Tairan Xu, Kai Zou, Edoardo Ponti, Luo Mai (2025). MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems. In NeurIPS.

PDF

Congjie He, Yeqi Huang, Pei Mu, Ziming Miao, Jilong Xue, Lingxiao Ma, Fan Yang, Luo Mai (2025). WaferLLM: Large Language Model Inference at Wafer Scale. In OSDI.

PDF Project

Marcel Wagenländer, Guo Li, Bo Zhao, Luo Mai, Peter Pietzuch (2024). Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections. In SOSP.

PDF

Chuanhao Sun, Zhihang Yuan, Kai Xu, Luo Mai, N Siddharth, Shuo Chen, Mahesh K Marina (2024). Learning high-frequency functions made easy with sinusoidal positional encoding. In ICML.

PDF Code

Yao Fu, Leyang Xue, Yeqi Huang, Andrei-Octavian Brabete, Dmitrii Ustiugov, Yuvraj Patel, Luo Mai (2024). ServerlessLLM: Low-Latency Serverless Inference for Large Language Models. In OSDI.

PDF Code Project

Jie Ren*, Xidong Feng*, Bo Liu*, Xuehai Pan*, Yao Fu, Luo Mai, Yaodong Yang (2023). TorchOpt: An Efficient Library for Differentiable Optimization. In JMLR.

PDF Code

Hanjing Wang*, Man-Kit Sit*, Congjie He, Ying Wen, Weinan Zhang, Jun Wang, Yaodong Yang, Luo Mai (2023). GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models. In ICML.

PDF Code

Chijun Sima*, Yao Fu*, Man-Kit Sit, Liyi Guo, Xuri Gong, Feng Lin, Junyu Wu, Yongsheng Li, Haidong Rong, Pierre-Louis Aublin, Luo Mai (2022). Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model Update. In USENIX OSDI.

PDF

Bo Liu, Xidong Feng, Jie Ren, Luo Mai, Rui Zhu, Haifeng Zhang, Jun Wang, Yaodong Yang (2022). A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning. In NeurIPS.

PDF

Jie Ren*, Wenteng Liang*, Ran Yan, Luo Mai, Shiwen Liu, Xiao Liu (2022). MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment. In ECCV.

PDF Code

See all publications