Article: link
The U.S. military nearly intercepted a Chinese vessel in the Middle East this spring after an intelligence report claimed the ship was carrying components for a nuclear weapons program, according to CNN.
The intelligence report circulated within the U.S. military during the war with Iran and prompted preparations to intercept the vessel. Four sources familiar with the episode told the news outlet that armed U.S. personnel were preparing to board the ship, while military aircraft were already in the air.
The operation was called off shortly after officials discovered that the intelligence report had been generated by an AI chatbot and contained inaccurate information.
A subsequent review found that an analyst at U.S. Special Operations Command Pacific in Hawaii had entered intelligence about the ship's manifest into the chatbot.
The chatbot combined open-source information with classified signals intelligence and drew conclusions about the material aboard the ship. The analyst then used AI again to format the findings into an official intelligence report that was disseminated through military channels.
It was not immediately clear whether the chatbot was a commercial system or a customized government tool.
One source told CNN the intelligence was "entirely false" and "almost started a war."
"The internal tools are mostly just copies of the commercial stuff wearing lipstick," a former senior U.S. official said.
U.S. Special Operations Command Pacific and the Pentagon have yet to comment on CNN's report.
The incident comes as the U.S. military expands its use of AI for intelligence analysis, targeting, operational planning and other tasks. Military officials have promoted the technology as a way to process large amounts of information and speed up battlefield decision-making.
Jake Steckler, a research scholar at GovAI and former U.S. Army officer, said AI uncertainty is particularly important when decisions could lead to the use of force.
"It's important for service members to understand the uncertainty inherent to LLMs," Steckler told TechCrunch. "But it's especially critical for any decisions that could lead to use of force, like targeting, intelligence analysis, or operational planning. There are life and death consequences for those decisions."
Steckler said the incident should lead to stronger safeguards rather than abandoning the technology.
"These tools can be useful in the right contexts and with the right safeguards in place," he said. "But prioritizing adoption speed over all else will likely lead to incidents that only make service members lose trust in these systems, which ultimately is only going to slow adoption."
Looks to me more like a failure of intel and analysts rather than AI.
Wolfhag
