ChatGPT's summary on an article I sent:
The article identifies a genuine and serious problem with AI-generated voter information. But its headline and presentation are more sensational than the underlying evidence warrants. In particular, it blurs the distinction between ChatGPT and other AI systems and between incorrect answers and failures to answer.
I found the original article, published today (November 2) by Joe Wilkins in Futurism, as well as the October 24 Washington Post investigation on which it relies.
1. What the Washington Post actually discovered
The Post tested six major AI chatbots, asking them to prepare voter guides for addresses in every American county. It analyzed more than 3,000 responses.
Among its findings:
• ChatGPT invented a U.S. Senate race in Missouri that wasn't taking place.
• Claude incorrectly identified local Wyoming politicians as candidates for federal office.
• The chatbots failed to identify a congressional district in 48% of responses.
• Among responses that did identify a district, 22% identified it incorrectly.
The Post also found that some chatbots repeated erroneous information from websites that themselves appeared to have been produced using AI.
These are substantive findings, not merely anecdotes.
2. The important statistical distinction
The Futurism article announces:
A nearly 48-percent error rate.
But that is not what the underlying study established.
The 48% figure measures how often chatbots failed to specify a congressional district, not how often they gave an incorrect district.
That distinction matters. An AI that says it cannot determine your district is behaving quite differently from one that confidently identifies the wrong district.
Of course, an incomplete voter guide is still a defective voter guide. But conflating omissions with false statements obscures an important distinction in reliability.
Moreover, the Post found that ChatGPT, Google AI Mode, and Perplexity appeared less error-prone on district identification partly because they frequently omitted the district altogether. That is hardly a ringing endorsement, but it is relevant to interpreting the results.
3. Does the article unfairly single out ChatGPT?
The headline certainly emphasizes ChatGPT, although the underlying investigation tested six different systems.
Nevertheless, ChatGPT demonstrably made mistakes, including inventing a Senate contest. There is no justification for dismissing those mistakes merely because competing systems also performed poorly.
The article's most serious substantive point is that an AI can deliver inaccurate information with the appearance of authority.
This is a particularly troubling combination: a polished, confident answer supported by citations that turn out to be unreliable.
The Post documented an especially revealing example. ChatGPT repeated an incorrect Georgia congressional district identification from a small, AI-generated voter-information website. The website's operator reportedly was surprised that ChatGPT had cited his experimental site.
That illustrates a problem larger than any single chatbot: AI-generated errors can circulate through the information ecosystem and return to other AI systems disguised as independent corroboration.
4. There is evidence the article leaves out
Other research provides a more complicated picture.
A 2026 study by the States United Democracy Center found that ChatGPT and Google's AI answers improved substantially between testing rounds. In its later sample of 402 responses, researchers found no verifiable factual errors, although answers were often incomplete and insufficiently directed voters toward official sources. This study asked different questions from the Post, so the results are not directly comparable.
Meanwhile, the Brennan Center for Justice found in August that major chatbots generally challenged familiar election conspiracy claims rather than accepting them. That same investigation exposed serious weaknesses in preventing the creation of deceptive election imagery.
In an October 22 follow-up, the Brennan Center reported that ChatGPT had made substantial improvements in its safeguards against creating realistic election-disinformation images.
Thus, the available evidence supports a distinction between several different problems: factual inaccuracies, incomplete information, responses to political falsehoods, and deliberate fabrication of deceptive material.
They should not all be lumped together under the general heading of misinformation.
5. The larger question: Can you trust ChatGPT?
There are two different kinds of trust involved.
First, factual reliability. Can ChatGPT accurately tell you who is running for Congress in your district? The Post investigation demonstrates that such answers require independent verification. Official state election websites are the appropriate authority.
Second, intellectual reliability. Can ChatGPT distinguish facts from interpretations, acknowledge uncertainty, and examine competing arguments without manipulating the evidence?
That second question is especially important when discussing history and politics.
You and I have discussed the Federalist, Andrew Jackson and Indian removal, slavery and the coming of the Civil War, and presidential evaluations. These subjects require more than retrieving facts. They require weighing evidence, considering alternative explanations, and distinguishing contemporary perceptions from historical judgments.
An AI can get the dates right and still mislead through selection, omission, or framing. Conversely, it can make an ordinary factual mistake without exhibiting ideological bias.
The Post investigation establishes significant factual unreliability in a particular task. It does not establish that the mistakes were systematically partisan or deliberately designed to influence voters. That would require another kind of investigation, examining the direction and distribution of errors.
My conclusion
I would separate three judgments:
• The underlying investigation: Substantial, methodical, and deserving of attention.
• The Futurism presentation: Based on real evidence, but its 48% characterization is misleading and its emphasis on ChatGPT obscures the multi-platform findings.
• The implications for AI: Serious. AI systems should not present uncertain election information as verified fact, particularly when official records are available.
There is also an irony here. The article criticizes AI for producing misleading information while itself presenting a statistic in a way that could lead readers to a mistaken conclusion.
That doesn't invalidate its central concern. It does illustrate why the same standards of accuracy should apply to journalists and AI alike.
One final observation: The most dangerous AI error may not be the ridiculous one, but the plausible one. An invented Senate race is relatively easy to catch. A subtle distortion of a historical argument or a selective presentation of political evidence can be much harder to detect.
That is where I think your longstanding insistence on primary sources, competing interpretations, and intellectual argument rather than received conclusions is particularly well placed.
Me:
This is actually a familiar phenomenon to me as a miniatures collector and painter. Errors get picked up and repeated sometimes until they become the norm. Of course getting the wrong color on a drummer's coat is not quite as important in the grand scheme of things!
ChatGPT:
Exactly. And military miniatures provide an unusually good illustration of how an error can acquire the appearance of authority through repetition.
Imagine that an illustrator in 1965 misinterprets a contemporary description of a uniform. A manufacturer uses that illustration as the basis for a figure. A respected painter reproduces the colors, and his photograph appears in a wargaming magazine. Other painters copy it. Eventually somebody writes a uniform guide incorporating the same mistake. Fifty years later, five apparently independent sources agree about the color of that drummer's coat.
But there was only one original source for the error, and everything else is derivative.
Historians have a name for a closely related phenomenon: circular reporting, or information laundering. The appearance of independent corroboration conceals a common origin.
AI introduces two complications.
First, speed and scale. An error that might once have taken decades to become established can now spread through thousands of websites and AI-generated summaries in weeks.
Second, the disappearance of provenance. When an AI combines information from numerous sources into a fluent paragraph, the reader may have no way of distinguishing an eyewitness account, a scholarly reconstruction, and somebody's imaginative guess.
There is another interesting parallel with miniature painting. Once an erroneous uniform becomes conventional, a historically accurate figure may actually look wrong to collectors accustomed to the mistake. Consensus has become the standard against which evidence is judged.
And that is a genuinely troubling possibility for AI: not merely that it repeats mistakes, but that repeated mistakes eventually become the material from which future systems learn what is supposedly true.
Your drummer's coat may be a small matter, but the epistemological problem is exactly the same.
