On Oct. 25, 2024, 10 days before the election that would return Republican Donald Trump to the White House and over three months after Kamala Harris ascended to the top of the Democratic ticket, ChatGPT was still telling users that Joe Biden was running for a second term as president.

Image

The “careless failure with profound democratic implications” is outlined in a recent study co-authored by a University of Massachusetts Amherst journalism scholar detailing how the generative AI model was serving up false information in the runup to the high-stakes election.

Less than two weeks before Election Day 2024, the study found that nearly 10% of election-related claims by ChatGPT were inaccurate, and that the model used “hedging,” “omission” and, to abbreviate a term used by the authors, “B.S.,” that obscured political realities. The research is published in the journal Information, Communication & Society.

Paper co-author Heesoo Jang, assistant professor of media law and ethics at UMass Amherst, says though ChatGPT may be able to ace tests and analyze large datasets, it was never designed to uphold democratic institutions—a role that framers of the Constitution recognized as essential when they enshrined a free press.

“We should be concerned and alarmed by the fact that while ChatGPT is nailing exams, it is also failing in these areas that it wasn’t designed for,” she says. “ChatGPT is commercially driven. It isn’t designed in a way that centers democracy.”

To test the model’s grasp of the 2024 election, Jang and researchers at the University of North Carolina at Chapel Hill (UNC-Chapel Hill) fed it both partisan and nonpartisan-framed prompts related to election information, disinformation and threats, candidates, and Project 2025. They took steps, including new “chats” and “incognito” web browsing, to ensure previous prompts did not influence later responses.

The researchers audited ChatGPT-4o mini, which was the default free version of ChatGPT available to users during the study period.

Of 592 verifiable claims it made on Oct. 25, 2024, 9.6% were inaccurate. In addition to claiming Biden was still in the race, ChatGPT inaccurately stated that North Carolina Gov. Roy Cooper was seeking reelection, even though he was term limited.

The study also shows the AI model hedged 18% of the time by returning answers with qualifying phrases like “could potentially,” “generally aligned” and “typically.” In other instances, it omitted critical information about efforts to overturn the 2020 election.

Then, there’s the category that researchers labelled B.S. They found ChatGPT appeared unable to recognize terms that partisan actors redefined and politicized, such as the “DEI hire” pejorative used against Kamala Harris. The model defined it as someone hired to oversee diversity, equity and inclusion efforts at an organization.

In another case, the chatbot made up the policy platforms of statewide school superintendent candidates in North Carolina, using generic language that appeared nearly identical.

“It was trying to fill in the gaps,” Jang explains. “It can’t give us, ‘I don’t know,’ because that’s not how it is designed.”

Though ChatGPT models have been updated since 2024 and would likely perform better today, she’s not optimistic about the 2026 midterm elections, particularly as AI increasingly replaces internet searches and links to primary sources.

“Truth and accurate electoral information—and also dealing with disinformation—are not things we can train large language models to handle using past datasets alone,” Jang asserts. “It will be really hard for them to catch up with what’s going on now, unless they are designed to prioritize democracy, which is extremely difficult when their development is driven primarily by commercial incentives.”

She argues that the rise of AI makes fact-based journalism even more essential.

“We need journalism that not only delivers truth, but also helps people evaluate the truth,” Jang says, by calling out mis- and disinformation, avoiding false equivalency and sounding the alarm when democratic norms are breached.

The research was co-authored by Shannon C. McGregor and Lorcan Neill of the Hussman School of Journalism and Media and the Center for Information, Technology, and Public Life at UNC-Chapel Hill.