HPCClusterScape: Increasing Transparency and Efficiency of Shared High-Performance Computing Clusters for Large-scale AI Models.

Heungsub Lee

Heungseok Park Aeree Cho Hyojun Jeon Hayoung Lee Youngil Yang Sungjae Lee Heungsub Lee Jaegul Choo HPCClusterScape: Increasing Transparency and Efficiency of Shared High-Performance Computing Clusters for Large-scale AI Models. 2023 abs/2310.02120 CoRR https://doi.org/10.48550/arXiv.2310.02120 db/journals/corr/corr2310.html#abs-2310-02120

Boseop Kim HyoungSeok Kim Sang-Woo Lee 0001 Gichang Lee Dong-Hyun Kwak Dong Hyeon Jeon Sunghyun Park 0005 Sungju Kim Seonhoon Kim Dongpil Seo Heungsub Lee Minyoung Jeong Sungjae Lee 0002 Minsub Kim SukHyun Ko Seokhun Kim Taeyong Park 0003 Jinuk Kim Soyoung Kang Na-Hyeon Ryu Kang Min Yoo Minsuk Chang Soobin Suh Sookyo In Jinseong Park Kyungduk Kim Hiun Kim Jisu Jeong Yong Goo Yeo Donghoon Ham Dongju Park Min Young Lee Jaewook Kang Inho Kang Jung-Woo Ha 0001 Woo-Myoung Park Nako Sung What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers. 3405-3424 2021 EMNLP (1) https://doi.org/10.18653/v1/2021.emnlp-main.274 https://aclanthology.org/2021.emnlp-main.274 conf/emnlp/2021-1 db/conf/emnlp/emnlp2021-1.html#KimKLLKJ0KKSLJL21

Boseop Kim HyoungSeok Kim Sang-Woo Lee 0001 Gichang Lee Dong-Hyun Kwak Dong Hyeon Jeon Sunghyun Park 0005 Sungju Kim Seonhoon Kim Dongpil Seo Heungsub Lee Minyoung Jeong Sungjae Lee 0002 Minsub Kim SukHyun Ko Seokhun Kim Taeyong Park 0003 Jinuk Kim Soyoung Kang Na-Hyeon Ryu Kang Min Yoo Minsuk Chang Soobin Suh Sookyo In Jinseong Park Kyungduk Kim Hiun Kim Jisu Jeong Yong Goo Yeo Donghoon Ham Dongju Park Min Young Lee Jaewook Kang Inho Kang Jung-Woo Ha 0001 Woo-Myoung Park Nako Sung What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers. 2021 abs/2109.04650 CoRR https://arxiv.org/abs/2109.04650 db/journals/corr/corr2109.html#abs-2109-04650

Chiheon Kim Heungsub Lee Myungryong Jeong Woonhyuk Baek Boogeon Yoon Ildoo Kim Sungbin Lim Sungwoong Kim torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models. 2020 abs/2004.09910 CoRR https://arxiv.org/abs/2004.09910 db/journals/corr/corr2004.html#abs-2004-09910

Woonhyuk Baek Minsuk Chang Aeree Cho Jaegul Choo Jung-Woo Ha 0001 Donghoon Ham Sookyo In Dong Hyeon Jeon Hyojun Jeon Jisu Jeong Minyoung Jeong Myungryong Jeong Inho Kang Jaewook Kang Soyoung Kang Boseop Kim Chiheon Kim Hiun Kim HyoungSeok Kim Ildoo Kim Jinuk Kim Kyungduk Kim Minsub Kim Seokhun Kim Seonhoon Kim Sungju Kim Sungwoong Kim SukHyun Ko Donghyun KwakDong-Hyun Kwak Gichang Lee Hayoung Lee Min Young Lee Sang-Woo Lee 0001 Sungjae Lee Sungjae Lee 0002 Sungbin Lim Dongju Park Heungseok Park Jinseong Park Sunghyun Park 0005 Taeyong Park 0003 Woo-Myoung Park Na-Hyeon Ryu Dongpil Seo Soobin Suh Nako Sung Youngil Yang Yong Goo Yeo Kang Min Yoo Boogeon Yoon