Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platforms.

Louis Feng

Zhongyi Lin Ning Sun Pallab Bhattacharya Xizhou Feng Louis Feng John D. Owens Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platforms. 226-238 2025 February 36 IEEE Trans. Parallel Distributed Syst. 2 https://doi.org/10.1109/TPDS.2024.3507814 db/journals/tpds/tpds36.html#LinSBFFO25 streams/journals/tpds

Shaochen (Henry) Zhong Yifan Lu Lize Shao Bhargav Bhushanam Xiaocong Du Yixin Wan Yucheng Shi Daochen Zha Yiwei Wang 0001 Ninghao Liu 0001 Kaixiong Zhou Shuai Xu Kai-Wei Chang 0001 Louis Feng Vipin Chaudhary Xia Hu 0001 MQuAKE-Remastered: Multi-Hop Knowledge Editing Can Only Be Advanced with Reliable Evaluations. 2025 ICLR https://openreview.net/forum?id=m9wG6ai2Xk conf/iclr/2025 db/conf/iclr/iclr2025.html#ZhongLSBDWSZWLZ25 Mingyu Liang Hiwot Tadese Kassa Wenyin Fu Brian Coutinho Louis Feng Christina Delimitrou Lumos: Efficient Performance Modeling and Estimation for Large-scale LLM Training. 2025 MLSys https://openreview.net/forum?id=mwEOLauKI5 https://proceedings.mlsys.org/paper_files/paper/2025/hash/a66caa1703fe34705a4368c3014c1966-Abstract-Conference.html conf/mlsys/2025 db/conf/mlsys/mlsys2025.html#LiangKFCFD25

Mingyu Liang Hiwot Tadese Kassa Wenyin Fu Brian Coutinho Louis Feng Christina Delimitrou Lumos: Efficient Performance Modeling and Estimation for Large-scale LLM Training. 2025 April abs/2504.09307 CoRR https://doi.org/10.48550/arXiv.2504.09307 db/journals/corr/corr2504.html#abs-2504-09307 streams/journals/corr

Zhongyi Lin Ning Sun Pallab Bhattacharya Xizhou Feng Louis Feng John D. Owens Towards Universal Performance Modeling for Machine Learning Training on Multi-GPU Platforms. 2024 abs/2404.12674 CoRR https://doi.org/10.48550/arXiv.2404.12674 db/journals/corr/corr2404.html#abs-2404-12674

Mingyu Liang Wenyin Fu Louis Feng Zhongyi Lin Pavani Panakanti Shengbao Zheng Srinivas Sridharan 0002 Christina Delimitrou Mystique: Enabling Accurate and Scalable Generation of Production AI Benchmarks. 37:1-37:13 2023 ISCA https://doi.org/10.1145/3579371.3589072 https://www.wikidata.org/entity/Q131130728 conf/isca/2023 db/conf/isca/isca2023.html#LiangFFLPZSD23 Daochen Zha Louis Feng Liang Luo Bhargav Bhushanam Zirui Liu 0001 Yusuo Hu Jade Nie Yuzhen Huang Yuandong Tian Arun Kejariwal Xia Hu 0001 Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models. 2023 MLSys https://proceedings.mlsys.org/paper_files/paper/2023/hash/2f982f26692c7fd45831598085f02588-Abstract-mlsys2023.html conf/mlsys/2023 db/conf/mlsys/mlsys2023.html#ZhaFLBLHNHTK023

Mingyu Liang Wenyin Fu Louis Feng Zhongyi Lin Pavani Panakanti Srinivas Sridharan 0002 Christina Delimitrou Mystique: Accurate and Scalable Production AI Benchmarks Generation. 2023 abs/2301.04122 CoRR https://doi.org/10.48550/arXiv.2301.04122 db/journals/corr/corr2301.html#abs-2301-04122

Daochen Zha Louis Feng Liang Luo Bhargav Bhushanam Zirui Liu 0001 Yusuo Hu Jade Nie Yuzhen Huang Yuandong Tian Arun Kejariwal Xia Hu 0001 Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models. 2023 abs/2305.01868 CoRR https://doi.org/10.48550/arXiv.2305.01868 db/journals/corr/corr2305.html#abs-2305-01868

Srinivas Sridharan 0002 Taekyung Heo Louis Feng Zhaodong Wang Matt Bergeron Wenyin Fu Shengbao Zheng Brian Coutinho Saeed Rashidi Changhai Man Tushar Krishna Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces. 2023 abs/2305.14516 CoRR https://doi.org/10.48550/arXiv.2305.14516 db/journals/corr/corr2305.html#abs-2305-14516

Zhongyi Lin Louis Feng Ehsan K. Ardestani Jaewon Lee John Lundell Changkyu Kim Arun Kejariwal John D. Owens Building a Performance Model for Deep Learning Recommendation Model Training on GPUs. 48-58 2022 HIPC https://doi.org/10.1109/HiPC56025.2022.00019 conf/hipc/2022 db/conf/hipc/hipc2022.html#LinFALLKKO22 Zhongyi Lin Louis Feng Ehsan K. Ardestani Jaewon Lee John Lundell Changkyu Kim Arun Kejariwal John D. Owens Building a Performance Model for Deep Learning Recommendation Model Training on GPUs. 227-229 2022 ISPASS https://doi.org/10.1109/ISPASS55109.2022.00030 conf/ispass/2022 db/conf/ispass/ispass2022.html#LinFALLKKO22 Daochen Zha Louis Feng Bhargav Bhushanam Dhruv Choudhary Jade Nie Yuandong Tian Jay Chae Yinbin Ma Arun Kejariwal Xia Hu 0001 AutoShard: Automated Embedding Table Sharding for Recommender Systems. 4461-4471 2022 KDD https://doi.org/10.1145/3534678.3539034 conf/kdd/2022 db/conf/kdd/kdd2022.html#ZhaFBCNTCMKH22 Daochen Zha Louis Feng Qiaoyu Tan Zirui Liu 0001 Kwei-Herng Lai Bhargav Bhushanam Yuandong Tian Arun Kejariwal Xia Hu 0001 DreamShard: Generalizable Embedding Table Placement for Recommender Systems. 2022 conf/nips/2022 NeurIPS http://papers.nips.cc/paper_files/paper/2022/hash/62302a24b04589f9f9cdd5b02c344b6c-Abstract-Conference.html db/conf/nips/neurips2022.html#ZhaFTLLBTKH22

Daochen Zha Louis Feng Bhargav Bhushanam Dhruv Choudhary Jade Nie Yuandong Tian Jay Chae Yinbin Ma Arun Kejariwal Xia Hu 0001 AutoShard: Automated Embedding Table Sharding for Recommender Systems. 2022 abs/2208.06399 CoRR https://doi.org/10.48550/arXiv.2208.06399 db/journals/corr/corr2208.html#abs-2208-06399

Daochen Zha Louis Feng Qiaoyu Tan Zirui Liu 0001 Kwei-Herng Lai Bhargav Bhushanam Yuandong Tian Arun Kejariwal Xia Hu 0001 DreamShard: Generalizable Embedding Table Placement for Recommender Systems. 2022 abs/2210.02023 CoRR https://doi.org/10.48550/arXiv.2210.02023 db/journals/corr/corr2210.html#abs-2210-02023

Xiaocong Du Bhargav Bhushanam Jiecao Yu Dhruv Choudhary Tianxiang Gao Sherman Wong Louis Feng Jongsoo Park Yu Cao 0001 Arun Kejariwal Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems. 1421-1428 2021 ICMLA https://doi.org/10.1109/ICMLA52953.2021.00229 conf/icmla/2021 db/conf/icmla/icmla2021.html#DuBYCGWFPCK21

Xiaocong Du Bhargav Bhushanam Jiecao Yu Dhruv Choudhary Tianxiang Gao Sherman Wong Louis Feng Jongsoo Park Yu Cao 0001 Arun Kejariwal Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems. 2021 abs/2105.01064 CoRR https://arxiv.org/abs/2105.01064 db/journals/corr/corr2105.html#abs-2105-01064

Sven Woop Louis Feng Ingo Wald Carsten Benthin Embree ray tracing kernels for CPUs and the Xeon Phi architecture. 44:1 2013 SIGGRAPH Talks https://doi.org/10.1145/2504459.2504515 conf/siggraph/2013talks db/conf/siggraph/siggraph2013talks.html#WoopFWB13 Ingrid Hotz Louis Feng Bernd Hamann Kenneth I. Joy Tensor-Fields Visualization Using a Fabric-like Texture Applied to Arbitrary Two-dimensional Surfaces. 139-155 2009 Mathematical Foundations of Scientific Visualization, Computer Graphics, and Massive Data Exploration https://doi.org/10.1007/b106657_8 books/sp/09/MHR2009 db/books/collections/MHR2009.html#HotzFHJ09

Louis Feng Ingrid Hotz Bernd Hamann Kenneth I. Joy Anisotropic Noise Samples. 342-354 2008 14 IEEE Trans. Vis. Comput. Graph. 2 https://doi.org/10.1109/TVCG.2007.70434 http://doi.ieeecomputersociety.org/10.1109/TVCG.2007.70434 https://www.wikidata.org/entity/Q46124550 db/journals/tvcg/tvcg14.html#FengHHJ08

Ingrid Hotz Louis Feng Hans Hagen Bernd Hamann Kenneth I. Joy Tensor Field Visualization Using a Metric Interpretation. 269-281 2006 Visualization and Processing of Tensor Fields https://doi.org/10.1007/3-540-31272-2_16 books/sp/06/WH2006 db/books/collections/WH2006.html#HotzFHHJ06

Serban D. Porumbescu Brian Budge Louis Feng Kenneth I. Joy Shell maps. 626-633 2005 24 ACM Trans. Graph. 3 https://doi.org/10.1145/1073204.1073239 https://doi.org/10.1145/1186822.1073239 db/journals/tog/tog24.html#PorumbescuBFJ05

Ingrid Hotz Louis Feng Hans Hagen Bernd Hamann Kenneth I. Joy Boris Jeremic Physically Based Methods for Tensor Field Visualization. 123-130 2004 conf/visualization/2004 IEEE Visualization https://doi.org/10.1109/VISUAL.2004.80 https://doi.ieeecomputersociety.org/10.1109/VISUAL.2004.80 http://dl.acm.org/citation.cfm?id=1034439 db/conf/visualization/visualization2004.html#HotzFHHJJ04 Jonathan I. Maletic Andrian Marcus Louis Feng Source Viewer 3D (sv3D) - A Framework for Software Visualization. 812-813 2003 conf/icse/2003 ICSE https://doi.org/10.1109/ICSE.2003.1201299 https://doi.ieeecomputersociety.org/10.1109/ICSE.2003.1201299 http://dl.acm.org/citation.cfm?id=776964 db/conf/icse/icse2003.html#MaleticMF03 Andrian Marcus Louis Feng Jonathan I. Maletic Comprehension of Software Analysis Data Using 3D Visualization. 105-114 2003 conf/iwpc/2003 IWPC https://doi.org/10.1109/WPC.2003.1199194 https://doi.ieeecomputersociety.org/10.1109/WPC.2003.1199194 db/conf/iwpc/iwpc2003.html#MarcusFM03 Andrian Marcus Louis Feng Jonathan I. Maletic 3D Representations for Software Visualization. 27-36 2003 conf/softvis/2003 SOFTVIS db/conf/softvis/softvis2003.html#MarcusFM03 https://doi.org/10.1145/774833.774837 Andrian Marcus Louis Feng Jonathan I. Maletic Source Viewer 3D (sv3D): A System for Visualizing Multi Dimensional Software Analysis Data. 62-63 2003 VISSOFT conf/vissoft/2003 db/conf/vissoft/vissoft2003.html#MarcusFM03 Ehsan K. Ardestani Carsten Benthin Matt Bergeron Pallab Bhattacharya Bhargav Bhushanam Brian Budge Yu Cao 0001 Jay Chae Kai-Wei Chang 0001 Vipin Chaudhary Dhruv Choudhary Brian Coutinho Christina Delimitrou Xiaocong Du Xizhou Feng Wenyin Fu Tianxiang Gao Hans Hagen Bernd Hamann Taekyung Heo Ingrid Hotz Xia Ben HuXia Hu 0001 Yusuo Hu Yuzhen Huang Boris Jeremic Kenneth I. Joy Hiwot KassaHiwot Tadese Kassa Arun Kejariwal Changkyu Kim Tushar Krishna Kwei-Herng Lai Jaewon Lee Mingyu Liang Zhongyi Lin Ninghao Liu 0001 Zirui Liu 0001 Yifan Lu John Lundell Liang Luo Yinbin Ma Jonathan I. Maletic Changhai Man Andrian Marcus Jade Nie John D. Owens Pavani Panakanti Jongsoo Park Serban D. Porumbescu Saeed Rashidi Lize Shao Yucheng Shi Srinivas Sridharan 0002 Ning Sun Qiaoyu Tan Yuandong Tian Ingo Wald Yixin Wan Yiwei Wang 0001 Zhaodong Wang Sherman Wong Sven Woop Shuai Xu Jiecao Yu Daochen Zha Shengbao Zheng Shaochen ZhongShaochen (Henry) Zhong Kaixiong Zhou