AI error nearly led US to seize Chinese ship: ‘Almost started a war’

A faulty intelligence report produced with the help of artificial intelligence and circulated throughout the U.S. military during the war in Iran nearly led the United States to launch a military operation against a Chinese vessel — and could have sparked a confrontation between the two powers, CNN reported Friday, citing people familiar with the incident.According to the report, the episode took place last spring, when an intelligence assessment reaching U.S. military officials concluded that a Chinese vessel in the Middle East was carrying components linked to a nuclear weapons development program. The information immediately raised serious concerns, prompting the military to begin preparing to intercept the ship.Gallery(Photo: Shutterstock, Aaron Schwartz / AFP, Evan Vucci/POOL/AFP, AP Photo / Jon Gambrell)Four sources who spoke with CNN said preparations had reached an advanced stage: Armed U.S. personnel were preparing to board the vessel and military aircraft were already in the air.Only shortly before the planned operation did officials decide to scrutinize the intelligence report more closely. They then discovered it had been prepared with the help of artificial intelligence and that a chatbot used by the analyst had incorrectly identified the material the ship was carrying.CNN said it was unable to determine what the vessel’s cargo actually contained.One source described the report as “entirely false,” adding: “It almost started a war.”According to CNN, U.S. action against a Chinese vessel could have triggered a serious escalation and potentially an armed confrontation between Washington and Beijing. The network said the episode illustrates a more immediate danger than the extreme scenarios senior technology executives have warned about in recent weeks: not AI “going out of control,” but humans making potentially momentous decisions based on false information it generates.The report was prepared by an analyst at Special Operations Command. According to CNN, the analyst entered intelligence about the ship’s cargo manifest into a chatbot, using information originating from U.S. Special Operations Command Pacific, which is headquartered in Hawaii.It was unclear whether the analyst used a commercial chatbot or a system developed for the U.S. government. The system combined information from open sources with classified signals intelligence stored in government databases and reached the incorrect conclusion about the cargo.The analyst then used artificial intelligence again, this time to turn the findings into the standard format of an official intelligence report — the kind of document military decision-makers are accustomed to treating as a reliable source — and circulated it.Defense Secretary Pete Hegseth; he pledged to remove bureaucratic barriers to artificial intelligence (Photo: Jeremias Gonzalez/AP Photo)A former senior U.S. official familiar with the artificial intelligence systems used by defense and intelligence agencies said internal government tools are not necessarily fundamentally different from commercial AI products.U.S. Special Operations Command Pacific and the Pentagon did not respond to CNN’s requests for comment.The episode comes as the U.S. military and intelligence community accelerate the adoption of artificial intelligence across nearly every aspect of their work, from analyzing vast quantities of intelligence and selecting targets to budgeting, logistics and supply chain management.The goal is to allow the military to make decisions more quickly, in part because of concerns that the United States could fall behind China and other rivals in the use of the technology.In January, Defense Secretary Pete Hegseth unveiled the Pentagon’s “AI Acceleration Strategy,” aimed at removing barriers and rapidly expanding the use of the technology so it would be available to roughly 3 million military personnel and civilian employees across all classification levels.The plan included projects to integrate AI into battlefield management, intelligence and decision-making.At the program’s launch, Hegseth said the United States would “unleash experimentation, eliminate bureaucratic barriers, focus our investments” to ensure U.S. leadership in military artificial intelligence.But U.S. officials said implementation of AI systems is taking place in a decentralized way, with different parts of the military and intelligence community using different tools under inconsistent instructions and safety standards.They said there is currently no single set of rules defining how AI-generated information must be verified before it is used in operational decision-making.Concern is particularly acute when it comes to target selection.A person familiar with military policy told CNN that the use of AI in targeting is “definitely something that is ramping up,” but said clear guidelines still do not exist to ensure that keeping a human in the decision-making process will actually prevent civilian casualties or friendly fire.The AI “hallucination” — the term used when an AI system produces information that sounds credible but is false — that led to the Chinese ship episode was not an isolated case, according to one source.Several veteran intelligence officials said the technology is increasing pressure on analysts to produce and distribute intelligence quickly, raising the risk of mistakes. They said younger analysts who have grown accustomed to using such tools may also be more inclined to trust them without sufficient scrutiny.“AI allows you to get to a bad idea faster,” one source said.Another sensitive China incident: F-35 parts reached Hong KongAlongside the faulty intelligence report, U.S. authorities have opened an investigation into another sensitive security incident involving China: A shipment of parts from an F-35 stealth fighter, which was being sent from Australia to the United States for repairs, was unexpectedly diverted to Hong Kong, Politico reported Friday.According to three people familiar with the matter, the components were being shipped on behalf of aircraft manufacturer Lockheed Martin through an intermediary, but during their journey across the Pacific the shipment arrived in Hong Kong instead of the United States.It remains unclear why the shipment’s route was changed or who currently has possession of the parts, according to the report.The Pentagon confirmed there was an issue involving a shipment of unserviceable F-35 components.The F-35 program office said it was working with U.S. authorities and industry partners “to recover the components, investigate the incident and implement safeguards to prevent a recurrence.”The incident has prompted an investigation on Capitol Hill and a series of briefings for congressional committees dealing with national security.According to the sources, State Department and Pentagon officials began briefing congressional staff on the matter in June, with additional briefings expected in the coming weeks.The main concern centers on the type of components included in the shipment.Two of the sources said the parts included an F-35 cockpit canopy containing technology related to the aircraft’s stealth capabilities that is available only to the United States and its allies.The canopy is a transparent acrylic shell covering the cockpit and is coated with a thin, transparent layer of radar-absorbing material designed to reduce the aircraft’s radar signature and make it more difficult for enemy systems to detect.U.S. officials are concerned that if such a component fell into the hands of an adversary, it could be closely examined and parts of the technology could potentially be reverse-engineered.A US military F-35 (Photo: REUTERS/Jana Rodenbusch/File Photo)The fact that the shipment reached Hong Kong has heightened concerns that China could gain access to the components.Beijing has long been accused of trying to obtain information about sensitive U.S. weapons programs, including the F-35.“It’s no secret that China has been aggressively trying to get F-35 data through electronic espionage, among other means,” J.J. Gertler, a former House Armed Services Committee staffer and military aviation analyst at the Congressional Research Service, told Politico.Gertler said that even if Beijing already possesses specifications or other information about the parts, having access to an actual component could allow it to verify and improve the information it already has.Lockheed Martin said it could not comment on the status of shipments for security reasons but stressed that it complies with all U.S. regulations governing the transfer of aircraft components.The company added that its personnel handle every shipment “with the utmost diligence and safeguards” to protect the security of the F-35 program and participating allies.The F-35 program is one of the largest and most expensive weapons programs in U.S. history. Along with supply chain problems and spare-parts shortages, it has for years struggled with high maintenance costs and difficulties meeting readiness targets for the aircraft fleet.The episode also highlights a longstanding problem in the F-35 program’s vast and complex supply chain, which relies on maintenance and repair facilities in multiple countries as well as commercial shipping companies.Countries participating in the program use a shared global pool of spare parts, which remain Pentagon property until they are installed on aircraft but are, in practice, managed largely by contractors.A 2023 report by the U.S. Government Accountability Office found that more than 1 million F-35 spare parts, worth tens of millions of dollars, had been classified as lost over roughly five years.The watchdog criticized the Pentagon’s limited ability to track the parts, in part because much of the relevant information is held by contractors that manage the supply system.