Three Knobs for Steering a Multi-Agent System
A Megagon Labs preprint names the handles a human gets in a coordinated LLM workflow, and the user study shows what staying in the loop actually costs.
Multi-agent AI systems look like a black box from the outside. The user sees a finished product: a report, a code change, a research brief. The orchestration underneath, where several LLM "agents" hand work to one another and assemble a plan, is invisible. The reasoning that produced the output is gone before the user reads the output.
That gap is not new. Pilots who trusted the autopilot on Air France 447 in 2009 could not see what the pitot tubes were doing, and the aircraft's automation masked a stall the crew never saw coming, as a case study of the accident from Sassofia describes in detail. The cockpit problem is now showing up in AI workflows. When a multi-agent LLM system goes off the rails, the human looking at the final answer often has no good way to find the place where it went wrong. The intermediate reasoning is gone. The plan is gone. The user is left with a verdict and a question: should I trust this?
A new preprint from Megagon Labs, the research group known for its data-centric AI work, tries to make that plan visible again, and gives the human three places to grab it. The paper, AMBIPOM: A Design Space for Human-LLM Co-Planning in Multi-Agent Systems, was accepted into the 2026 Companion of the ACM Collective Intelligence Conference (CAIS). It formalizes a design space for human-in-the-loop workflows along three axes:
These are not abstract categories. The authors built a prototype that exposes them as a working interface, then ran a user study and a controlled LLM-revision benchmark to see how people actually use the handles. The headline finding: users converged on hybrid semantic-plus-structural workflows. The hybrid mode is not a free lunch. The paper surfaces a real effort-control-risk trade-off curve. More control costs more effort. Less effort means less precise control, and more risk of a downstream plan that surprises the user.
That trade-off is the constructive payload of the paper. Multi-agent systems are not a category that auto-justifies trust, and the paper does not claim they do. The argument is narrower and more useful: humans currently supervise these systems at the outcome level, where their leverage is weakest. AMBIPOM proposes process-level supervision as the interface. A human who can see the plan, and reshape it, has a different relationship to the system than a human who can only approve the output.
The user study in the paper is small by production standards. Megagon Labs reports the prototype as a research artifact, and the code and data are open-sourced on GitHub. The companion LLM-revision benchmark is a controlled evaluation, not a field deployment. The work is preprint, accepted at a workshop-style companion venue rather than a top-tier proceedings track, and the design space is a research contribution, not a finished standard. Readers evaluating the paper should weight it accordingly.
What changes for teams if process-level supervision becomes a first-class interface? Two things, both visible in the AMBIPOM prototype.
First, plan ownership becomes a real design question. In an outcome-only workflow, the plan is implicit and the human is the approver. In a process-aware workflow, the plan is a shared object, and the question of who can edit which axis becomes a product and policy decision. Some teams will want humans to control scope but not mode. Some will invert it. The design space is the menu, not the recipe.
Second, error localization gets cheaper. When a multi-agent system produces a bad output, finding the bad step in the plan is half the debugging. AMBIPOM's targeted-axis editing is the mechanism that makes this tractable, by giving the human a way to pin a node and let the rest of the plan update. The paper's benchmark shows the LLM revision step is workable. The open question is how it scales beyond prototype-scale data and user sessions.
The surprise problem is not solved by the design space. It is given a handle. Adjacent work, such as arXiv 2509.24826 on multi-agent coordination, points in the same direction from a different angle, though it is a preprint and the connection is by theme rather than by direct citation. What AMBIPOM contributes is the vocabulary. The three axes are now legible. The effort-control-risk trade-off is now legible. Readers can use both when they evaluate the next multi-agent tool a vendor pitches, or when they design one of their own.
The bet implicit in the paper is that process-level supervision will become a first-class interface the same way version control did for source code. A research contribution is not a guarantee. The design space is open, the prototype is on GitHub, and the trade-off is named. That is a usable starting point.