Abstract
ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
| Original language | English |
|---|---|
| Pages (from-to) | 81-108 |
| Number of pages | 28 |
| Journal | Issues in Accounting Education |
| Volume | 38 |
| Issue number | 4 |
| DOIs | |
| State | Published - Nov 2023 |
| Externally published | Yes |
Keywords
- GPT-3
- accounting education
- artificial intelligence
- assessment
- chat bots
- crowdsourcing
Fingerprint
Dive into the research topics of 'The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions?'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver
}
In: Issues in Accounting Education, Vol. 38, No. 4, 11.2023, p. 81-108.
Research output: Contribution to journal › Article › peer-review
TY - JOUR
T1 - The ChatGPT Artificial Intelligence Chatbot
T2 - How Well Does It Answer Accounting Assessment Questions?
AU - Wood, David A.
AU - Achhpilia, Muskan P.
AU - Adams, Mollie T.
AU - Aghazadeh, Sanaz
AU - Akinyele, Kazeem
AU - Akpan, Mfon
AU - Allee, Kristian D.
AU - Allen, Abigail M.
AU - Almer, Elizabeth D.
AU - Ames, Daniel
AU - Arity, Viktor
AU - Barr-Pulliam, Dereck
AU - Basoglu, K. Asli
AU - Belnap, Andrew
AU - Bentley, Jeremiah W.
AU - Berg, Terje
AU - Berglund, Nathan R.
AU - Berry, Erica
AU - Bhandari, Avishek
AU - Bhuyan, Md Nazmul Hasan
AU - Black, Paul W.
AU - Blondeel, Eva
AU - Bond, David
AU - Bonrath, Annika
AU - Borthick, A. Faye
AU - Boyle, Erik S.
AU - Bradford, Marianne
AU - Brandon, Duane M.
AU - Brazel, Joseph F.
AU - Brockbank, Bryan G.
AU - Burger, Marcus
AU - Byzalov, Dmitri
AU - Cannon, James N.
AU - Caro, Cecil
AU - Carr, Abraham H.
AU - Cathey, Jack
AU - Cating, Ryan
AU - Charron, Kimberly
AU - Chavez, Stacy
AU - Chen, Jason
AU - Chen, Jennifer C.
AU - Chen, Jennifer W.
AU - Cheng, Christine
AU - Cheng, Xu
AU - Christensen, Brant E.
AU - Church, Kimberly Swanson
AU - Cicone, N. J.
AU - Constance, Patience
AU - Cooper, Lauren A.
AU - Correia, Candice L.
AU - Coyne, Joshua
AU - Cram, W. Alec
AU - Curtis, Asher
AU - Daigle, Ronald J.
AU - Dannemiller, Steve
AU - Davenport, Stephan A.
AU - Dawson, Gregory S.
AU - De Meyst, Karen J.L.
AU - Dell, Scott
AU - Demirkan, Sebahattin
AU - Denison, Christine A.
AU - Desai, Hrishikesh
AU - Desimone, Steven
AU - Diehl, Leah M.
AU - Dimes, Ruth
AU - Dong, Bei
AU - Donnelly, Amy
AU - Du Pon, Adam
AU - Duan, Huijue Kelly
AU - Duffey, Ada
AU - Dunn, Ryan T.
AU - Durkin, Mary P.
AU - Dzuranin, Ann C.
AU - Eberle, Rachel M.
AU - Ege, Matthew S.
AU - El Mahdy, Dina
AU - Esplin, Adam
AU - Eulerich, Marc
AU - Everaert, Patricia
AU - Farah, Nusrat
AU - Farish, Lauren
AU - Favere-Marchesi, Michael
AU - Fayard, Dutch
AU - Filosa, Jessica R.
AU - Ford, Melinda
AU - Franz, Diana R.
AU - Fulmer, Bachman P.
AU - Fulmer, Sarah
AU - Furner, Zhan Z.
AU - Gantman, Sonia
AU - Garner, Steve
AU - Garrett, Jace
AU - Geng, Xin
AU - Golden, Joanna
AU - Goldman, William
AU - Gomez, Josue
AU - Gooley, Mark
AU - Granitto, Shawn P.
AU - Green, Karen Y.
AU - Greenman, Cindy L.
AU - Gupta, Gaurav
AU - Guymon, Ronald N.
AU - Hale, Kevin
AU - Harper, Christopher J.
AU - Hartt, S. Allen
AU - Hawk, Holly
AU - Hawkins, Steven R.
AU - Hawkins, Erin M.
AU - Hay, David C.
AU - Heinzelmann, Rafael
AU - Henderson, Cassy D.
AU - Hendricks, Bradley E.
AU - Heninger, William G.
AU - Hill, Mary S.
AU - Holden, Nicole
AU - Holderness, D. Kip
AU - Holt, Travis P.
AU - Hoopes, Jeffrey L.
AU - Hsieh, Sheng Feng
AU - Huang, Feiqi
AU - Huang, Hua Wei
AU - Huang, Ting Chiao
AU - Huels, Brian W.
AU - Hunter, Kara
AU - Hurley, Patrick J.
AU - Inger, Kerry
AU - Islam, Sharif
AU - Ison, Isaac
AU - Issa, Hussein
AU - Jackson, Andrew B.
AU - Jackson, Scott C.
AU - Janvrin, Diane J.
AU - Jimenez, Peggy D.
AU - Johanson, Daniel
AU - Judd, J. Scott
AU - Kawada, Brett S.
AU - Kelton, Andrea Seaton
AU - Kern, Sara
AU - Kerr, Jon N.
AU - Keune, Marsha B.
AU - Kim, Mindy Hyo Jung
AU - Knox, Brian D.
AU - Kogan, Gregory
AU - Kotb, Amr
AU - Krane, Ronja
AU - Kremin, Joleen
AU - Krieg, Kimberly S.
AU - Kugel, Jonathan
AU - Kulset, Ellen M.
AU - Kuruppu, Chamara
AU - Laduca, Garrison
AU - Lamberton, Barbara A.
AU - Lamboy-Ruiz, Melvin A.
AU - Lang, Bradley
AU - Larocque, Stephannie A.
AU - Larson, Melissa P.
AU - Lawson, Bradley P.
AU - Lawson, James G.
AU - Lee, Lorraine
AU - Lenk, Margarita M.
AU - Li-Kuehne, Michelle
AU - Liljegren, Jonathan
AU - Lin, Yi Hung
AU - Liu, Wu Po
AU - Liu, Zishang
AU - Lock, Brandon
AU - Long, James H.
AU - Loraas, Tina
AU - Lowensohn, Suzanne
AU - Loy, Thomas R.
AU - Lyngstadaas, Hakim
AU - Maas, Wim
AU - Macgregor, Jason E.
AU - Madsen, Dag Øivind
AU - Malone, Carissa L.
AU - Margolin, Maximilian
AU - Marshall, Mary E.
AU - Martin, Rachel M.
AU - Mpofu, Colleen Mcclain
AU - McCoy, Chris
AU - McGuigan, Nicholas C.
AU - McSwain, Dwayne N.
AU - Meckfessel, Michele D.
AU - Mellon, Mark J.
AU - Melton, Olivia S.
AU - Mercado, Julie M.
AU - Mitsuda, Steven
AU - Modugu, Kennedy
AU - Moehrle, Stephen
AU - Chaghervand, Amirali Moeini
AU - Moffitt, Kevin
AU - Moon, Joon Seok
AU - Muehlmann, Brigitte
AU - Murray, Johnna
AU - Mwaungulu, Emmanuel S.
AU - Myers, Noah
AU - Naegle, J. Conrad
AU - Ndicu, Martin J.
AU - Nelson, Aaron S.
AU - Nguyen, Anh L.
AU - Niederkofler, Thomas
AU - Nikbakht, Ehsan
AU - O’Brien, Ann D.
AU - Ogunade, Kehinde M.
AU - O’Leary, Daniel
AU - Oler, Mitchell J.
AU - Oler, Derek K.
AU - Olsen, Kari Joseph
AU - Otalor, John I.
AU - Outlaw, Kyle W.
AU - Ozlanski, Michael E.
AU - Parlier, Jenny
AU - Paterson, Jeffrey S.
AU - Pearson, Christopher A.
AU - Petersen, Michael J.
AU - Petra, Steven T.
AU - Pickard, Matthew D.
AU - Pickerd, Jeffrey
AU - Pinsker, Robert
AU - Plante, Catherine
AU - Plecnik, James M.
AU - Price, Richard A.
AU - Quick, Linda A.
AU - Raedy, Jana
AU - Raschke, Robyn
AU - Ravenscraft, Julie
AU - Richardson, Vernon
AU - Rixom, Brett A.
AU - Robertson, John F.
AU - Rock, Iyad
AU - Romney, Miles A.
AU - Rozario, Andrea
AU - Ruff, Michael F.
AU - Rupley, Kathleen
AU - Saeedi, Ali
AU - Saiewitz, Aaron
AU - Salzsieder, Leigh W.
AU - Sarkar, Sayan
AU - Saulls, Michael
AU - Scanlan, Tialei A.
AU - Schaefer, Tammie J.
AU - Schaupp, Daniel
AU - Schneider, Gary P.
AU - Seebeck, Andreas
AU - Sellers, R. Drew
AU - Seto, Samantha C.
AU - Sevel, Romi Lee
AU - Shan, Yuxin
AU - Sherwood, Matthew G.
AU - Singorahardjo, Maggie
AU - Skaftadottir, Hanna Kristin
AU - Skomra, Justyna
AU - Smith, Jason L.
AU - Smith, Dallin O.
AU - Smith, James
AU - Snow, Mason C.
AU - Sommerfeldt, Ryan D.
AU - Sorensen, Kate B.
AU - Sorensen, Trevor L.
AU - Spieler, Andrew C.
AU - Stallings, Matthew A.
AU - Stallings, Lesya
AU - Stancill, Alan
AU - Stanley, Jonathan D.
AU - Stefaniak, Chad M.
AU - Stephens, Nathaniel M.
AU - Stewart, Bryan W.
AU - Stratopoulos, Theophanis C.
AU - Street, Daniel A.
AU - Subedi, Meena
AU - Summers, Scott L.
AU - Sundkvist, Charlotte H.
AU - Synn, Christina
AU - Tadesse, Amanuel
AU - Tapis, Gregory P.
AU - Tassin, Kerri
AU - Taylor, Samantha
AU - Teal, Mary
AU - Teeter, Ryan
AU - Tharapos, Meredith
AU - Theis, Jochen C.
AU - Thomas, Jack
AU - Thompson, Kristen S.
AU - Thornock, Todd A.
AU - Tietz, Wendy
AU - Travalent, Anthony M.
AU - Trinkle, Brad S.
AU - Truelson, J. Mike
AU - Turner, Michael C.
AU - Vagner, Brandon
AU - Vakilzadeh, Hamid
AU - van der Geest, Jesse
AU - van Pelt, Victor
AU - Vandervelde, Scott D.
AU - Vega, Jose
AU - Vera-Munoz, Sandra
AU - Villanueva, Brigham
AU - Vincent, Nishani Edirisinghe
AU - Wagener, Martin
AU - Walton, Stephanie
AU - Warne, Rick C.
AU - Watanabe, Olena V.
AU - Watson, David
AU - Watson, Marcia Weidenmier
AU - Weber, Jill
AU - Weirich, Thomas
AU - West, Ashley N.
AU - Wilford, Amanda L.
AU - Wilson, Aaron B.
AU - Winrow, Brian
AU - Winrow, Timothy
AU - Winrow, Tasia S.
AU - Wiseman, Denise
AU - Witte, Annie L.
AU - Wood, Bryan D.
AU - Wood, Jessica
AU - Woolley, Darryl
AU - Wright, Nicole S.
AU - Wu, Juan
AU - Xiong, Xiaomei Grazia
AU - Yatsenko, Dimitri
AU - Yazzie, Courtney E.
AU - Young, Glen M.
AU - Zhang, Chanyuan
AU - Zimmerman, Aleksandra B.
AU - Zoet, Emily
N1 - Publisher Copyright: © 2023, American Accounting Association. All rights reserved.
PY - 2023/11
Y1 - 2023/11
N2 - ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
AB - ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.
KW - GPT-3
KW - accounting education
KW - artificial intelligence
KW - assessment
KW - chat bots
KW - crowdsourcing
UR - http://www.scopus.com/inward/record.url?scp=85166407026&partnerID=8YFLogxK
U2 - 10.2308/ISSUES-2023-013
DO - 10.2308/ISSUES-2023-013
M3 - Article
AN - SCOPUS:85166407026
SN - 0739-3172
VL - 38
SP - 81
EP - 108
JO - Issues in Accounting Education
JF - Issues in Accounting Education
IS - 4
ER -