San Francisco startup Andon Labs says Drone-Bench aims to spur AI-safety debate. A Sept. 10 video showed an AI-guided quadcopter identify and follow a person in its office; OpenAI’s GPT-6 Astra and Anthropic’s Fable 5.1 each scored above 90%.
The team equipped a low-cost, off-the-shelf quadcopter with facial recognition and supplied it with videos of the office. It compared human-written code with code from large language models across five tasks: mapping, locating and navigating the drone, identifying a person and following them. Co-founder Lukas Petersson said Andon Labs does not make autonomous drone weapons and has no Defense Department contracts.
Critics question the safety framing. University of Washington linguist Emily M. Bender said focus on existential AI risks could benefit AI firms and distract from current harms. Autonomous-weapons advocate Peter Asaro said AI coding makes surveillance drones easier to build and urged accountable human oversight; Andon Labs’ website calls human-in-the-loop safety a “mirage.” CNN reported the U.S. military nearly intercepted a Chinese ship based on an AI-assisted report falsely saying it carried components for a nuclear weapons program. Bloomberg reported the military was changing AI targeting after a strike killed 175, mostly schoolchildren, at an Iranian girls’ school.
