{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T08:38:07Z","timestamp":1773218287251,"version":"3.50.1"},"reference-count":67,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T00:00:00Z","timestamp":1738886400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Manage. Inf. Syst."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>Large language models have been advancing very rapidly and are making substantial impacts on all areas of business and management. We review the development of large language models and their applications in business and management, and identify the major issues and challenges faced by both practitioners and researchers. Based on our review, we propose an agenda for information systems researchers on large language models and discuss some of the potential directions for future research. Lastly, we present the articles in the special issue as exemplary research on large language models and discuss their implications.<\/jats:p>","DOI":"10.1145\/3713032","type":"journal-article","created":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T12:50:46Z","timestamp":1738932646000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["An IS Research Agenda on Large Language Models: Development, Applications, and Impacts on Business and Management"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4579-4329","authenticated-orcid":false,"given":"Michael","family":"Chau","sequence":"first","affiliation":[{"name":"Faculty of Business and Economics, The University of Hong Kong, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5615-9967","authenticated-orcid":false,"given":"Jennifer","family":"Xu","sequence":"additional","affiliation":[{"name":"Computer Information Systems, Bentley University, Waltham, United States"}]}],"member":"320","published-online":{"date-parts":[[2025,2,7]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"T. Abuelsaad D. Akkil P. Dey A. Jagmohan A. Vempaty and R. Kokku. 2024. Agent-E: from autonomous web navigation to foundational design principles in agentic systems. arXiv:2407.13032. Retrieved from https:\/\/arxiv.org\/abs\/2407.13032"},{"key":"e_1_3_3_3_2","unstructured":"A. Askell Y. Bai A. Chen D. Drain D. Ganguli T. Henighan A. Jones N. Joseph B. Mann N. DasSarma et al. 2021. A general language assistant as a laboratory for alignment. arXiv:2112.00861. Retrieved from https:\/\/arxiv.org\/abs\/2112.00861"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2024.114172"},{"key":"e_1_3_3_5_2","unstructured":"Y. Bai A. Jones K. Ndousse A. Askell A. Chen N. DasSarma D. Drain S. Fort D. Ganguli T. Henighan et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback arXiv:2204.05862. Retrieved from https:\/\/arxiv.org\/abs\/2204.05862"},{"issue":"3","key":"e_1_3_3_6_2","first-page":"1433","article-title":"Managing artificial intelligence","volume":"45","author":"Berente N.","year":"2021","unstructured":"N. Berente, B. Gu, J. Recker, and R. Santhanam. 2021. Managing artificial intelligence. MIS Quarterly 45, 3 (2021), 1433\u20131450.","journal-title":"MIS Quarterly"},{"key":"e_1_3_3_7_2","unstructured":"X. Bi D. Chen G. Chen S. Chen D. Dai C. Deng H. Ding K. Dong Q. Du Z. Fu et al. 2024. DeepSeek LLM: Scaling open-source language models with longtermism. arXiv:2401.02954. Retrieved from https:\/\/arxiv.org\/abs\/2401.02954"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.techsoc.2023.102370"},{"key":"e_1_3_3_9_2","unstructured":"Botco.ai. 2023. Nearly three-quarters of all marketing departments use generative AI tools botco.ai survey finds. Retrieved January 14 2025 from https:\/\/www.prweb.com\/releases\/nearly-three-quarters-of-all-marketing-departments-use-generative-ai-tools-botco-ai-survey-finds-863152119.html"},{"key":"e_1_3_3_10_2","unstructured":"Cole. 2024. Most powerful LLMs. CodingScope. Retrieved January 14 2025 from https:\/\/codingscape.com\/blog\/most-powerful-llms-large-language-models"},{"key":"e_1_3_3_11_2","unstructured":"J. Devlin M.-W. Chang K. Lee and K. Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved January 14 2025 from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_3_12_2","unstructured":"S. Fatemi and Y. Hu. 2023. A comparative analysis of fine-tuned LLMs and few-shot learning of LLMs for financial sentiment analysis. arXiv:2312.08725. Retrieved from https:\/\/arxiv.org\/abs\/2312.08725"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42001-024-00250-1"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12599-023-00834-7"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.futures.2021.102884"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocad177"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.44720"},{"key":"e_1_3_3_18_2","unstructured":"D. Hendrycks C. Burns S. Basart A. Critch J. Li D. Song and J. Steinhardt. 2023. Aligning AI with shared human values. arXiv:2008.02275. Retrieved from https:\/\/arxiv.org\/abs\/2008.02275"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1111\/1911-3846.12832"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.54097\/30r2kk80"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2024\/18512"},{"key":"e_1_3_3_22_2","unstructured":"J. Kaplan S. McCandlish T. Henighan T. B. Brown B. Chess R. Child S. Gray A. Radford J. Wu and D. Amodei. 2020. Scaling laws for neural language models. arXiv:2001.08361. Retrieved from https:\/\/arxiv.org\/abs\/2001.08361"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.econlp-1.4"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1080\/0960085X.2021.1927212"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2023.102716"},{"key":"e_1_3_3_26_2","unstructured":"P. Lewis E. Perez A. Piktus F. Petroni V. Karpukhin N. Goyal H. K\u00fcttler M. Lewis W. Yih T. Rockt\u00e4schel S. Riedel and D. Kiela. 2021. Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv:2005.11401. Retrieved from https:\/\/arxiv.org\/abs\/2005.11401"},{"key":"e_1_3_3_27_2","unstructured":"Y. Li H. Wen W. Wang X. Li Y. Yuan G. Liu J. Liu W. Xu X. Wang Y. Sun et al. 2024. Personal LLM agents: Insights and survey about the capability efficiency and security. arXiv:2401.05459. Retrieved from https:\/\/arxiv.org\/abs\/2401.05459"},{"key":"e_1_3_3_28_2","unstructured":"H. Li W. Su C. Wang Y. Wu Q. Ai and Y. Liu. 2023. THUIR@COLIEE 2023: Incorporating structural knowledge into pre-trained language models for legal case retrieval. arXiv:2305.06812. Retrieved from https:\/\/arxiv.org\/abs\/2305.06812"},{"key":"e_1_3_3_29_2","unstructured":"L. Li Y. Zhou Z. Gao W. Hua L. Fan H. Yu L. Hangen Y. Zhang T. L. Assimes L. Hemphill and S. Ma. 2024. A scoping review of using large language models (LLMs) to investigate electronic health records (EHRs). arXiv:2405.03066. Retrieved from https:\/\/arxiv.org\/abs\/2405.03066"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3488832"},{"key":"e_1_3_3_31_2","unstructured":"S. Lundberg and S. I. Lee. 2017. A unified approach to interpreting model predictions. arXiv:1705.07874. Retrieved from https:\/\/arxiv.org\/abs\/1705.07874"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","unstructured":"X. Ma R. Zhao Y. Liu C. Deng and D. Du. 2024. Design of a large language model for improving customer service in telecom operators. Electronics Letters 60 10 Article e13218 (2024) 7 pages. 10.1049\/ell2.13218","DOI":"10.1049\/ell2.13218"},{"key":"e_1_3_3_33_2","volume-title":"A Future that Works: AI, Automation, Employment, and Productivity","author":"Manyika J.","year":"2017","unstructured":"J. Manyika, M. Chui, M. Miremadi, J. Bughin, K. George, P. Willmott, and M. Dewhurst. 2017. A Future that Works: AI, Automation, Employment, and Productivity. McKinsey Global Institute Research."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbah.2023.100022"},{"key":"e_1_3_3_36_2","unstructured":"H. Naveed A. U. Khan S. Qiu M. Saqib S. Anwar M. Usman N. Akhtar N. Barnes and A. Mian. 2024. A comprehensive overview of large language models. arXiv:2307.06435. Retrieved from https:\/\/arxiv.org\/abs\/2307.06435"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597307"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2023.0159"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.3390\/info15110662"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.25300\/MISQ\/2024\/790"},{"key":"e_1_3_3_41_2","article-title":"Forthcoming. Language models for online depression detection: A review and benchmark analysis on remote interviews","author":"Qin R.","unstructured":"R. Qin, R. Cook, K. Yang, A. Abbasi, D. Dobolyi, S. Seyedi, E. Griner, H. Kwon, R. Cotes, Z. Jiang, and G. Clifford. Forthcoming. Language models for online depression detection: A review and benchmark analysis on remote interviews. ACM Transactions on Management Information Systems.","journal-title":"ACM Transactions on Management Information Systems"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.5555\/3177663.3177664"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tbench.2023.100140"},{"key":"e_1_3_3_44_2","unstructured":"Salesforce. 2023. New research: 60% of marketers say generative AI will transform their role but worry about accuracy. Retrieved January 14 2025 from https:\/\/www.salesforce.com\/news\/stories\/generative-ai-for-marketing-research\/"},{"key":"e_1_3_3_45_2","unstructured":"A. M. Sami Z. Rasheed K.-K. Kemell M. Waseem T. Kilamo M. Saari A. N. Duc K. Syst\u00e4 and P. Abrahamsson. 2024. System for systematic literature review using multiple AI agents: Concept and an empirical evaluation. arXiv:2403.08399. Retrieved from https:\/\/arxiv.org\/abs\/2403.08399"},{"key":"e_1_3_3_46_2","doi-asserted-by":"crossref","first-page":"127979","DOI":"10.3389\/frai.2023.1279794","article-title":"The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal texts","volume":"6","author":"Savelka J.","year":"2023","unstructured":"J. Savelka and K. D. Ashley. 2023. The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal texts. Frontiers in Artificial Intelligence 6, Article 127979.","journal-title":"Frontiers in Artificial Intelligence"},{"key":"e_1_3_3_47_2","unstructured":"M. Schreiner. 2023. GPT-4 architecture datasets costs and more leaked. The Decoder. Retrieved January 14 2025 from https:\/\/thedecoder.com\/gpt-4-architecture-datasets-costs-and-more-leaked\/"},{"key":"e_1_3_3_48_2","unstructured":"N. Shazeer A. Mirhoseini K. Maziarz A. Davis Q. Le G. Hinton and J. Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv:1701.06538. Retrieved from https:\/\/arxiv.org\/abs\/1701.06538"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/AIIoT61789.2024.10578990"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1002\/aaai.12135"},{"key":"e_1_3_3_51_2","unstructured":"T. Tu A. Palepu M. Schaekermann K. Saab J. Freyberg R. Tanno A. Wang B. Li M. Amin N. Tomasev et al. 2024. Towards conversational diagnostic AI. arXiv:2401.05654. Retrieved from https:\/\/arxiv.org\/abs\/2401.05654"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-71089-9"},{"key":"e_1_3_3_53_2","article-title":"Forthcoming. Learning personalized privacy preference from public data","author":"Wang W.","unstructured":"W. Wang and B. Li. Forthcoming. Learning personalized privacy preference from public data. Information Systems Research.","journal-title":"Information Systems Research"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3661145"},{"key":"e_1_3_3_55_2","unstructured":"O. Weller B. Chang S. MacAvaney K. Lo A. Cohan B. V. Durme D. Lawrie and L. Soldaini. 2024. FollowIR: Evaluating and teaching information retrieval models to follow instructions. arXiv:2403.15246. Retrieved from https:\/\/arxiv.org\/abs\/2403.15246"},{"key":"e_1_3_3_56_2","article-title":"Forthcoming. Zero-shot construction of Chinese medical knowledge graph with GPT-3.5-turbo and GPT-4","author":"Wu L.","unstructured":"L. Wu, Y. Su, and G. Li. Forthcoming. Zero-shot construction of Chinese medical knowledge graph with GPT-3.5-turbo and GPT-4. ACM Transactions on Management Information Systems.","journal-title":"ACM Transactions on Management Information Systems"},{"key":"e_1_3_3_57_2","unstructured":"S. Wu O. Irsoy S. Lu V. Dabravolski M. Dredze S. Gehrmann P. Kambadur D. Rosenberg and G. Mann. 2023. BloombergGPT: A large language model for finance. arXiv:2303.17564. Retrieved from https:\/\/arxiv.org\/abs\/2303.17564"},{"key":"e_1_3_3_58_2","unstructured":"J. Wulf and J. Meierhofer. 2024. Utilizing large language models for automating technical customer support. arXiv:2406.01407. Retrieved from https:\/\/arxiv.org\/abs\/2406.01407"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-021-00043-6"},{"key":"e_1_3_3_60_2","unstructured":"J. Xie Z. Chen R. Zhang X. Wan and G. Li. 2024. Large multimodal agents: A survey. arXiv:2402.15116. Retrieved from https:\/\/arxiv.org\/abs\/2402.15116"},{"key":"e_1_3_3_61_2","unstructured":"J. Xie K. Zhang J. Chen T. Zhu R. Lou Y. Tian Y. Xiao and Y. Su. 2024. TravelPlanner: A benchmark for real-world planning with language agents. arXiv:2402.01622. Retrieved from https:\/\/arxiv.org\/abs\/2402.01622"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1287\/isre.2022.0128"},{"key":"e_1_3_3_63_2","doi-asserted-by":"crossref","unstructured":"H. Yang X.-Y. Liu and C. D. Wang. 2023. FinGPT: Open-source financial large language models. arXiv:2306.06031. Retrieved from https:\/\/arxiv.org\/abs\/2306.06031","DOI":"10.2139\/ssrn.4489826"},{"key":"e_1_3_3_64_2","article-title":"KETCH: A knowledge-enhanced transformer-based approach to suicidal ideation detection from social media content","author":"Zhang D.","unstructured":"D. Zhang, L. Zhou, J. Tao, T. Zhu, and G. Gao. Forthcoming. KETCH: A knowledge-enhanced transformer-based approach to suicidal ideation detection from social media content. Information Systems Research.","journal-title":"Information Systems Research"},{"key":"e_1_3_3_65_2","unstructured":"H. Zhao Z. Liu Z. Wu Y. Li T. Yang P. Shu S. Xu H. Dai L. Zhao H. Jiang et al. 2024. Revolutionizing finance with LLMs: An overview of applications and insights. arXiv:2401.11641. Retrieved from https:\/\/arxiv.org\/abs\/2401.11641"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1080\/07421222.2024.2415772"},{"key":"e_1_3_3_67_2","unstructured":"Y. Zhu Z. Wang J. Gao Y. Tong J. An W. Liao E. M. Harrison L. Ma and C. Pan. 2024. Prompting large language models for zero-shot clinical prediction with structured longitudinal electronic health record data. arXiv:2402.01713. Retrieved from https:\/\/arxiv.org\/abs\/2402.01713"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.3004555"}],"container-title":["ACM Transactions on Management Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3713032","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3713032","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:09Z","timestamp":1750295889000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3713032"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,7]]},"references-count":67,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3713032"],"URL":"https:\/\/doi.org\/10.1145\/3713032","relation":{},"ISSN":["2158-656X","2158-6578"],"issn-type":[{"value":"2158-656X","type":"print"},{"value":"2158-6578","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,7]]},"assertion":[{"value":"2025-01-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}