Budapest Post

Cum Deo pro Patria et Libertate
Budapest, Europe and world news

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
AI Disclaimer: An advanced artificial intelligence (AI) system generated the content of this page on its own. This innovative technology conducts extensive research from a variety of reliable sources, performs rigorous fact-checking and verification, cleans up and balances biased or manipulated content, and presents a minimal factual summary that is just enough yet essential for you to function as an informed and educated citizen. Please keep in mind, however, that this system is an evolving technology, and as a result, the article may contain accidental inaccuracies or errors. We urge you to help us improve our site by reporting any inaccuracies you find using the "Contact Us" link at the bottom of this page. Your helpful feedback helps us improve our system and deliver more precise content. When you find an article of interest here, please look for the full and extensive coverage of this topic in traditional news sources, as they are written by professional journalists that we try to support, not replace. We appreciate your understanding and assistance.
Newsletter

Related Articles

0:00
0:00
Close
Bangkok Ranked World's Top City for Remote Work in 2025
Satirical Sketch Sparks Political Spouse Feud in South Korea
Indonesia Quarry Collapse Leaves Multiple Dead and Missing
South Korean Election Video Pulled Amid Misogyny Outcry
Asian Economies Shift Away from US Dollar Amid Trade Tensions
Netflix Investigates Allegations of On-Set Mistreatment in K-Drama Production
US Defence Chief Reaffirms Strong Ties with Singapore Amid Regional Tensions
Vietnam Faces Strategic Dilemma Over China's Mekong River Projects
Malaysia's First AI Preacher Sparks Debate on Islamic Principles
Meta and Anduril Collaborate on AI-Driven Military Augmented Reality Systems
Russia's Fossil Fuel Revenues Approach €900 Billion Since Ukraine Invasion
Alcohol Industry Faces Increased Scrutiny Amid Health Concerns
U.S. Goods Imports Plunge Nearly 20% Amid Tariff Disruptions
Italy Faces Population Decline Amid Youth Emigration
Trump Accuses China of Violating Trade Agreement
OpenAI Faces Competition from Cheaper AI Rivals
Foreign Tax Provision in U.S. Budget Bill Alarms Investors
Russia Accuses Serbia of Supplying Arms to Ukraine
Gerry Adams Wins Libel Case Against BBC
EU Central Bank Pushes to Replace US Dollar with Euro as World’s Main Currency
U.S. Health Secretary Ends Select COVID-19 Vaccine Recommendations
Trump Warns Putin Is 'Playing with Fire' Amid Escalating Ukraine Conflict
India and Pakistan Engage Trump-Linked Lobbyists to Influence U.S. Policy
U.S. Halts New Student Visa Interviews Amid Enhanced Security Measures
Trump Administration Cancels $100 Million in Federal Contracts with Harvard
SpaceX Starship Test Flight Ends in Failure, Mars Mission Timeline Uncertain
King Charles Affirms Canadian Sovereignty Amid U.S. Statehood Pressure
EU Majority Demands Hungary Reverse Anti-LGBTQ+ Laws
Top Hotel Picks for 2025 Stays in Budapest Revealed
Iron Maiden Unveils 2025 Tour Setlist in Budapest
Chinese Film Week Opens in Budapest to Promote Cultural Exchange
Budapest Airport Launches Direct Flights to Shymkent
Von der Leyen Denies Urging EU Officials to Skip Budapest Pride
Alcaraz and Sinner Advance with Convincing Wins at Roland Garros
EU Ministers Lack Consensus on Sanctioning Hungary Over Rule of Law
EU Nations Urge Action Against Hungary's Pride Parade Ban
Putin's Helicopter Reportedly Targeted by Ukrainian Drones
U.S. Considers Withdrawing Troops from Europe
Russia Deploys Motorbike Squads in Ukraine Conflict
Critics Accuse European Court of Human Rights of Overreach
Spain Proposes 100% Tax on Non-EU Holiday Home Purchases
German Intelligence Labels AfD as Far-Right Extremist
Geert Wilders Threatens Dutch Coalition Over Migration Policy
Hungary Faces Multiple Challenges Amid EU Tensions and Political Shifts
Denmark Increases Retirement Age to 70, Setting a European Precedent
Any trade deal with US must be based on respect not threats', says EU commissioner
UK Leads in Remote Work Adoption, Averaging 1.8 Days a Week
Thirteen Killed in Russian Attacks Across Ukraine
High-Profile Incidents and Political Developments Dominate Global News
Netanyahu Accuses Western Leaders of 'Emboldening Hamas'
×