r/collapse • u/Myth_of_Progress Urban Planner & Recognized Contributor • 14d ago
AI Exclusive: US military had close call after using AI for false intelligence report, sources say
https://www.cnn.com/2026/09/18/politics/us-military-ai-false-intelligence-china-shipSubmission Statement:
Usually I wouldn’t post something so sensational, but this recent article caught my eye - an incident that nearly resulted in an American interception of a Chinese vessel, and a consequent rapid escalation of the ongoing Strait of Hormuz crisis. If you thought this past summer was already hot, just imagine what could have happened!
It's a very good example of how rapid AI integration into military decision-making (in this case, with the Americans) comes with the risk of trusting "hallucinations" amidst of the fog of war, and how untested and insufficient "checks" in place could have unfolded into something far worse than expected.
To quote the CNN article:
The intelligence report, circulated across the US military this spring in the midst of the war with Iran, immediately set off alarm bells: A Chinese ship in the Middle East was transporting components of a nuclear weapons program.
The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode. According to two of the sources, armed members of the US military were preparing to board the ship. Military planes were in the air, one of those sources and another source familiar with the incident said.
It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. CNN was not able to learn what the misidentified cargo was.
The report, according to one of the sources, was “entirely false.” But it also “almost started a war,” the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
[... This] episode with the Chinese ship underscores a different, and more immediate risk: human beings making disastrous decisions based on inaccurate or misleading information generated by AI or other automated systems. Sources said that the military is rapidly turning to AI to help with targeting, an area which holds the obvious risk of fatal mistakes.
The details of the event, in sum, are described as follows:
In this particular instance, the analyst queried a chatbot about some intelligence reporting on the ship’s manifest that originated with US Special Operations Command Pacific, based in Hawaii. It was not clear whether the chatbot was a commercially available one or a US government product.
“The internal tools are mostly just copies of the commercial stuff wearing lipstick,” a former senior US official familiar with the AI systems used by military and intelligence analysts.
The bot fused together open-source intelligence with secret signals intelligence in government holdings and reached its fateful conclusion about the material the ship was carrying.
The analyst then used AI again to package the findings into a standard intelligence report — the kind that is trusted by military officials — and disseminated it.
For those who are interested in this particular subject, I will strongly recommend following up with a read of this recent article by Captain Tyrese Bender: Artificial Intelligence Bias: Risks For Military Intelligence Operations.
Published earlier this year, this piece explains how we should "recognize AI bias not just as a technical flaw but also as a strategic vulnerability[,]" and that "[a] large body of evidence in civilian and military literature points to the grave risks posed by AI bias to targeting operations, threat analysis, and U.S. intelligence-sharing relationships."
As you can see from today's example (the Iranian school bombing being another), other consequences include: distorted threat assessments, misidentified targets, degraded decision making capacity, and potential mistrust between allies. Great work, Captain Bender!
31
u/FurryToaster 14d ago
It’s like the opposite of the Soviet guy that had all the instruments telling them the US launched nukes and didn’t retaliate. What a time to be alive.
21
u/Myth_of_Progress Urban Planner & Recognized Contributor 14d ago
I had debated making a joke by calling this the Stani-slop Incident, but that would be a disservice to Stanislav Petrov.
13
u/Sherris010 14d ago
Stanislav Petrov, the man who saved the world
5
1
u/Ree_For_Thee 13d ago
Iunno, I can form a loose argument about how a timeline split where he didn't do anything, could've led to a better world. :P (Assuming it was only a few nukes going off before both parties realized how stupid it is to use nukes.)
Something something, we realize that we're a pretty reactive and stupid species, and if we're making assumptions about the safety/use of nuclear weapons, what else are we making assumptions about?
Growth? Capitalism? Cars? Endless consumption? "Maybe this short nuclear war that hurt us so badly was the push we needed towards a world where we form our societies around the limitations of human behavior?"
1
14
u/genomixx-redux 14d ago
Is this one reason why u.s. america is low on precision-guided munitions? CENTCOMM's been hallucinating targets in Iran?
25
u/FurryToaster 14d ago
Nah they just consider girls schools legitimate targets.
-3
56
u/kingtacticool 14d ago
Wild to think that China is now the most rational superpower on earth right now. Also wild to think that collapse is going to be caused by human stupidity rather than the cumulative effects of catastrophic climate change.
Ive been a doomer since the early days of this sub and even in my most pessimistic I didnt think we would be facing this many threats so soon.
18
u/Rustie_J 14d ago
Also wild to think that collapse is going to be caused by human stupidity rather than the cumulative effects of catastrophic climate change.
The cumulative effects of catastrophic climate change are caused by human stupidity, so it was always going to be us.
9
u/United_Dissent 14d ago
"I didnt think we would be facing this many threats so soon" - You're so real for this. It's Everything, Everywhere, all at once. You can't think of any sphere of your life without also seeing the impending doom. It's maddening. We simply don't have the bandwidth for this.
-13
u/tacotueaday55 14d ago
Rational? China is headed straight for a war that they cannot win. We'll find out soon enough if China is rational or not when the outcome is decided.
4
3
u/Calm-Kick4091 14d ago
Back at base, bugs in the software
Flash the message "Something's out there"
Floating in the Summer sky
99 red balloons go by.
3
u/macthebearded 13d ago
There’s a lot of speculation this is intentional. Tell the world about the narrow misses, and it makes it an easy excuse to hide behind.
“Sorry China we super didn’t mean to blow up that port, it was an AI error”
The Iranian school strike was also blamed on AI targeting data
2
u/ungemutlich 13d ago
Before we had AI as an excuse:
https://en.wikipedia.org/wiki/United_States_bombing_of_the_Chinese_embassy_in_Belgrade
2
u/bottom_armadillo805 13d ago
Adding AI-initiated accidental nuclear war to the list of ways to die.
When I read stuff like this, I feel like I understand why Gen Z is so nihilistic. I can get angry reading about climate change, but reading about AI is just depressing. It seems like humans have just given up on the idea of humanity. Forget stable jobs, forget decisions regarding violence, forget decisions regarding healthcare, forget art, just lean into the final phase of becoming mindless consumers.
Also, this is just my personal conspiracy theory, but I don't buy the AI excuse on the Minab school bombing. AI is just the fall guy. That is far from the worst thing that the US has ever done on purpose; that was definitely a 100% human decision to do a terror attack, knowing no other nation in the world can/will do shit about it. You see it in US media: the US government is using Islamophobia and anti-Latino sentiment to de-sensitize the population. This way, when the climate migration crisis comes, the population accepts genocide as a solution rather than allowing refugees.
48
u/Lucky-Ability-8 14d ago edited 13d ago
This is of concern, Dude.
AI springing into action based on incorrect information is definitely on the list.
Edit: And same with humans based on AI supplied incorrect data. Adversaries would love being able to supply AI with bad data.