A Drone Killed Three Ukrainians. It Was Guided Entirely by A.I.
“A Russian drone with an Nvidia module autonomously targeted a gas station in Zaporizhzhia on July 6, killing Tetiana Bubynets and two others. Experts and commanders attribute the strike to experimental artificial intelligence selecting targets without human control.
An attack by what Ukrainian officials said was a Russian drone with an Nvidia chip presages a dystopian future of weaponry untethered to humans.
The young woman ran for her life, but it was too late.
A small Russian drone resembling a model airplane swooped down from the sky last month in the city of Zaporizhzhia in southeastern Ukraine. As it tried to navigate toward a gas station, it crashed into a wall and exploded in a hail of shrapnel, killing Tetiana Bubynets, 19, a university student, and two others.
They were among the thousands of Ukrainian civilians who have died in Russian airstrikes, but the circumstances of their deaths came with an ominous twist. The drone that killed them was guided by an experimental artificial intelligence system, not a human pilot, according to drone experts, Ukrainian military commanders and the forensic team that examined wreckage from the deadly strike and others in the city.
The drone was programmed by human operators to travel toward a particular gas station, but once it got close, it chose its exact target — most likely propane tanks — on its own, based on its training to recognize such tanks and strike them, the experts and military officials said.
It is a development long watched for in the war in Ukraine, presaging a dystopian future in which killer robots roam the skies making life-or-death decisions that reduce targets to data points.
“This is a risk for the whole world,” said Col. Serhiy Minaiev, the commander of air defenses in Zaporizhzhia. “In a few years, we will be living in a ‘Terminator’ movie. It’s no joke. Machines are making decisions to strike.”
Analysis of debris from this attack and others in Zaporizhzhia found that the drones had onboard minicomputers, sold commercially by Nvidia, that made their targeting decisions, the experts and military officials said. Nvidia produces a majority of the chips powering the world’s most advanced A.I. systems.
The presence of Nvidia modules and a lack of antennas on the drones led Ukrainian air defense commanders to surmise that the weapons were guided by an autonomous A.I. system, which was confirmed on further investigation.
Autonomous A.I. systems can be trained on thousands of images to recognize categories like “river,” “military truck” or “person.” Self-targeting drones use their cameras to hunt for these things with greater precision than a human pilot can provide. But opponents of such weapons argue that, without human intervention, drones may be more prone to mistakes or violations of the laws of war, such as by failing to distinguish innocent civilians from combatants.
The use of self-targeting A.I. in the gas station attack, on July 6, would be the first documented case in which civilian deaths were caused by a Russian drone outfitted with such a system, said Kateryna Bondar, a senior fellow at the Wadhwani AI Center at the Center for Strategic and International Studies in Washington. She added that the Nvidia module “is the best proof” that Russia was experimenting with autonomous A.I.
While an adviser to President Volodymyr Zelensky has written on social media that Russia was testing self-targeting A.I. systems, the fatalities have not been previously reported, nor have the details of the Nvidia Jetson Orin computer module that was used.
Both Russia and Ukraine were already using A.I. in multiple drone models before the fully autonomous drones emerged. In those earlier models, A.I. is used only for what is known as the “last mile” — the final stage of the attack, after a remote human pilot has selected the target. Fully autonomous A.I. removes humans from the final target selection.
The Nvidia chip in the Zaporizhzhia attack was not encrypted, Ukrainian officials said, so they could see the terrain images that had been uploaded to the module, enabling the drone to track visual landmarks and stay on course. They were also able to examine coding, they said, that revealed what types of targets the drone had been trained to recognize and attack, like propane tanks.
Earlier drone models are generally either piloted by human operators, through radio transmissions or fiber-optic cables, or they are preprogrammed with coordinates. As drones have taken over the battlefield in Ukraine, their pilots have become prime targets.
An autonomous A.I.-guided drone, by contrast, requires no human pilots and can be more accurate than a munition programmed to hit coordinates.
Local air defense officials say that Russia has been aiming for civilian objects like pumps or propane tanks at gas stations in trying out autonomous drones in Zaporizhzhia, along with military recruitment centers. It is not clear why these targets were chosen for the tests.
Vadym Kushnikov, a drone expert at the Kharkiv Aviation Institute in Ukraine, said the drone that killed the three civilians was “a preprogrammed tool trained in virtual reality to track specific objects.”
While traditional software follows step-by-step rules written by a person, an A.I. system derives its own rules by finding patterns across enormous amounts of data, a process called machine learning. That allows it to manage unanticipated situations. It also means no one can fully predict what it might do.
Rights groups and the International Red Cross vehemently oppose the use of A.I. weapons without humans in the loop. Such groups say that autonomous systems essentially leave robots to interpret international law in determining legitimate targets in war.
But some exports argue that, in the future, weapons should in fact be required to have A.I. systems. These weapons are safer, the argument goes, because they are able to make decisions before striking, even if not always perfectly, unlike unguided aerial bombs or ordinary artillery shells that just fall without seeing what they will hit.
The weapon in the fatal strike was flying in a group of about half a dozen drones, according to the regional military governor, the city’s air defense commander and an air defense team on duty that day. None were emitting radio signals, and they had other characteristics associated with Russia’s autonomous A.I. tests, including being launched in a wave, the officials said.
While some of the A.I.-guided drones tested by Russia have detonated on impact, others have not and remained intact, Ukrainian officials said. They showed debris from two such drones to reporters for The New York Times. The Times photographed the Nvidia computer module inside the drones and sent the images to the company.
Nvidia confirmed that the photos showed a Jetson Orin minicomputer. Such modules, which cost only a few hundred dollars, have been found on several types of Russian weapons.
In a statement, the company said that the minicomputer was a consumer-grade product “sold to students, developers and start-ups for a wide range of beneficial applications.” Nvidia sells the computer in many countries but not in Russia. It is, though, widely available through resellers, the statement said.
Ukraine, too, has tested self-targeting A.I. systems. Mykhailo Fedorov, the defense minister who was recently ousted, said in an interview with The Timeslate last month that Ukraine tested an entirely autonomous A.I. system in Russian-occupied Crimea over the past several months. Mr. Fedorov said that the devices targeted fuel storage sites and military equipment and that no civilians had been killed.
While the drones in the Zaporizhzhia attack had no antennas to send back data on the tests, Russian forces often fly reconnaissance drones that can record strikes. The city is also believed to harbor Russian spies who report battlefield damage assessments.
The absence of an antenna would not by itself prove that a drone was self-targeting. It could be preprogrammed to fly a route, with computer algorithms like those used for decades in auto-piloting systems for aircraft. But that drone would not need an Nvidia chip.
Russia began test flights of A.I.-guided Molniya drones — a mass-produced fixed-wing model — in May, and started testing self-targeting systems in the V2U, another fixed-wing drone, several months before that, Ukrainian air defense experts said. Some of the V2U tests were only for “last mile” targeting.
In interviews, some residents of Zaporizhzhia, which lies just nine miles from the front, said that the thought of devices buzzing over the city, making their own decisions to kill, brought chills. But Ukrainian cities have been so heavily bombarded for years that life had already been reshaped by less sophisticated drones.
Interceptor drones, piloted (for now) by men and women who are local heroes, are one answer to the drone threat. Anti-drone nets are another, covering about 240 miles of roads in and around Zaporizhzhia.
Another defensive measure is camouflage. Plastic sheeting, for example, is propped at odd angles around propane tanks at gas stations, in an effort to fool A.I. image recognition systems.
Moments before the exploding drone barreled toward the gas station on July 6, a group of people huddled by the wall of a nearby apartment building for safety, having already heard blasts as the drones came in waves.
Oksana Ishenko, a station attendant, stood beside Ms. Bubynets. “She was crying from fear,” Ms. Ishenko recalled.
Then some in the group started to shout, “A plane! A plane!” Ms. Ishenko said. A drone was heading their way. Ms. Bubynets and several others bolted, fanning out in a parking lot. After the drone failed to make its way around the apartment building, it detonated before reaching its target.
Ms. Bubynets, who had been studying accounting at Zaporizhzhia National University, lay on the pavement, unmoving, with blood flowing from her nose. She was declared dead at a hospital, said Tetiana Melikhova, one of her professors. The two others killed, Oleksiy Svirin, 41, and Roman Karpiy, 48, died later of their wounds.
“They are training the technology,” Colonel Minaiev, the city’s air defense commander, said of the Russians. A.I. is good but not yet perfect, he added. “They are looking for mistakes and will fix them.”
Evelina Riabenko, Stanislav Kozliuk and Paul Mozur contributed reporting.
Andrew E. Kramer is the Kyiv bureau chief for The Times, who has been covering the war in Ukraine since 2014.“
No comments:
Post a Comment