A modular AI pipeline scores 89% on cluttered dishes by stacking four swappable vision models

A robot arm in a lab kitchen picks a plate out of a sink and slides it into a dishwasher rack without any kitchen-specific training. The trick is not a smarter arm. It is a four-slot perception stack where every visual step is a separate foundation model that the team can replace when a better one ships.

The system is the subject of arXiv preprint 2608.04042 from the RAIV Lab at the University of Washington. The paper proposes a plug-and-play perception stack for household robots: detection, segmentation, 3D feature extraction, and 6D pose estimation are each handled by its own model, and the pipeline can be re-tuned by swapping one module at a time.

On the authors' own 20-scene cluttered kitchen benchmark, the best four-module combination posts an 89.12% Average Distance to the Inlier (ADI) on dishware. ADI is the paper's measure of how close a predicted 6D pose is to the ground-truth pose. A lower distance is better, and the 89.12% is the share of predictions that pass the authors' own success threshold on their own scenes. The score is not directly comparable to results on public benchmarks like YCB-V or Tabletop, and the paper's own caveats note that the test set is lab-collected.

What the modular design buys is something the field has been trying to bolt on after the fact: a way for the robot to inherit the field's progress. A foundation model, in the loose sense the paper uses, is a large pre-trained AI that adapts to many tasks. The kitchen-robot pipeline leans on four of them.

The four slots, in the order the robot uses them. Open-vocabulary detection runs on LLMDet, a detector that uses a language model to name objects it has never been trained on. The team points it at a sink full of mixed dishware and asks it to find "plate" or "mug" without a closed label list. Multi-view segmentation runs on SAMv2, the follow-up to Meta's Segment Anything. SAMv2 lifts the 2D masks from each camera into a shared 3D view so the robot sees a plate as one object across angles, even when a hand is in the way.

Instance-aware 3D reconstruction fuses those SAMv2 masks with depth and a feature backbone. 6D pose estimation and grasp planning rely on a 2D-to-3D feature pipeline that pairs DINOv2, Meta's general-purpose visual feature model, with GeoTransformer, a geometric matching network trained to align partial 3D point clouds.

The "best" configuration, LLMDet plus SAMv2 plus DINOv2 plus GeoTransformer, is the one that maximizes ADI on the 20-scene test. The paper uses that score to identify which foundation model to plug into each slot, not as a benchmark win in the public sense. The authors treat the score closer to an ablation: which detector, which segmenter, which 3D feature, which pose estimator does the most work.

The two real-world demonstrations are narrower than the benchmark suggests. The first is the sink-to-dishwasher transfer shown in the project page videos: the arm grasps plates and cups from a cluttered sink and loads them into a rack. The second is cup stacking, where the robot builds a tower of cups in a kitchen setting. Both were recorded on a fixed arm with cameras around the workspace, and the paper does not report quantitative success rates for either demo.

That matters for the read. A reader who stops at "89% on cluttered dishes" is liable to assume a deployable home robot. The honest read is narrower: a four-model perception stack, evaluated on 20 controlled kitchen scenes by its own authors, with two dishware demos that look clean but lack published success counts. The pipeline is the story. The number is one data point in choosing which foundation model to put in each slot.

The compounding claim is the one that survives those caveats. Each module is a piece of independent research, and LLMDet, SAMv2, DINOv2, and GeoTransformer are all maintained by other teams. Each can be replaced when a successor ships. The kitchen robot does not need to be retrained to pick up a better detector. That is the practical pitch: as the foundation-model field improves, the robot improves with it, one slot at a time.

The paper does not claim general-purpose household behavior, and the demos do not include non-dishware items, soft objects, or unstructured rooms. What it does support, with the modular design in place, is a perception system that ages with the field rather than freezing at publication.

The next release from Meta, or a new 3D matching network from the geometric learning community, becomes a candidate upgrade to one of the four slots. The RAIV Lab's contribution is the seat for it.