AINEWS
Andon Labs' Drone Bench scores how well AI models can write the code that turns a consumer drone into an office floor tracker, measured against a human who leaned on a coding assistant.
A consumer drone entered a small office, found the person in a reference photo, and kept them in frame as they walked through the building. The code that flew it was not written by a human pilot. It was written by a frontier AI model, and the result was scored against a reference implementation written by a person who leaned on a coding assistant to get there.
That handoff, from model-written code to a real drone doing a real job in a real room, is the unit of progress on Drone-Bench, the first public capability test built around a concrete physical-world surveillance chain on off-the-shelf hardware. Released this week by Andon Labs, the benchmark does not ask whether a model can answer trivia about drones. It asks whether the model can write the code that makes a drone do five things, in order, without a human in the loop.
The five tasks are not a checklist. They are a pipeline.
Reconstruct flies a pre-programmed path through the office and turns the footage into a 3D model of the room, a 2D obstacle map, and per-frame camera poses. Localize takes a single query image and places it inside that model so the drone knows where it is. Navigate plans a path on the obstacle map and re-calls Localize during flight to stay oriented. Detect takes a reference face photo and returns bounding boxes around the matching face in each new frame. Follow drives the drone from those boxes, keeping the person in frame as they move.
Each downstream task consumes the artifacts from the task before it. A model that nails Reconstruct but fumbles the pose handoff can break Localize, which breaks Navigate, which means the drone never gets close enough for Detect to return useful boxes. The score that matters is the chain, not any one of its links.
Andon Labs made two choices that bound the claim. The eval is held out by design: no AI lab can train on it, which is the difference between a capability test and a leaderboard that rewards memorization. And the human baseline, the reference code that scores are measured against, was written by a person who used a coding assistant. That is the bar a model has to clear, and it is a real bar. It is also a specific one: cooperative lighting, a single floor, a willing subject, an office, and a reference photo the system was given ahead of time.
What the eval deliberately leaves out is the part that would matter most in a real surveillance chain. There is no adversarial target, no occluded face, no counter-surveillance, no operator decision about when the drone is allowed to look, no legal or consent layer, and no multi-floor or outdoor handoff. Andon Labs frames the release as evidence that "physical autonomy of frontier models is accelerating." That is the publisher's view, not a neutral finding, and it is consistent with what the eval actually measures: today's models can plausibly write the code to do an office-floor tracking job on consumer hardware under friendly conditions.
The open question is not whether the next model can pass Reconstruct. It is what the next version of this benchmark will have to add. Harder environments and adversarial targets are obvious. The less obvious layer is everything outside the code: who decides when the drone flies, who it can be pointed at, and what counts as a fair test of a capability that, in the real world, is bounded less by the model and more by the operator and the law. The pipeline is now legible. The next thing worth scoring is the part Drone-Bench chose to leave out.