An empirical audit — 33 controlled experiments, negative results included

The Control Gap

The industry is scaling robot foundation models a thousand-fold and pricing the result for the datacenter. We ran the controlled experiments the demos skip, and found the opposite economics: in the closed loop, precision emerges at the small end — from real action data, a second camera, and a controller light enough for the cheapest terminal on the shelf.

M LAB · 3-SEED OR CONTROLLED A/B THROUGHOUT · HARDWARE: TWO CONSUMER RTX 3090s, TOTAL

Frontier world models, running on the cheapest terminals, with high-precision intelligence emerging at the edge — not rented from a datacenter.

Fig. 1 — The industry climbs the y-axis; the loop is won near the bottom

robot policy & VLA model sizes at release, public figures · log scale · hover any point

Grey: published robot foundation models, 2022–2026 — four orders of magnitude of scaling, most of it deployable only on datacenter GPUs. Teal: the controller that wins our closed-loop benchmark outright (Fig. 2), at 1.3M parameters — small enough for a $249 embedded board with room to spare. The bet embedded in this chart is the entire note: parameter count is the industry's proxy for progress, and in the control loop the proxy fails.⁰

Fig. 2 — Closed-loop pick success: one environment, one diagnosis at a time

ManiSkill 3 PickCube · pixels-to-torques · 25 evaluation episodes per rung · scripted-expert demonstrations

How to read it: every rung is the same task and evaluation protocol; one variable changes between adjacent rungs. Model-predictive control in pixel latent space never succeeds. A 5B vision-language-action model and a 3M behavior-cloning transformer tie at 4%. The winner is a 1.3M tracking controller whose only upgrades are a better camera and more demonstrations — no architecture change after rung five, and the curve had not saturated when we stopped.

Over four campaigns we ran thirty-three controlled experiments across seven open world and policy models, two real robot datasets, and one closed-loop simulator — all of it on two consumer GPUs. This note reports the results we consider decision-grade: where prediction quality provably dissociates from control; why manufactured action labels cannot substitute for real ones; which levers actually move closed-loop success and which merely move benchmarks; and what the measured economics imply for where intelligence should live — on the robot, or in someone else's cloud. Every headline number has a from-scratch control arm and a do-nothing baseline. The experiments that failed are reported next to the ones that worked.

§1

Prediction and control are different properties — a natural experiment proves it

The field's implicit bet is that a model which predicts video well will control robots well. We found a clean natural experiment to test this: an open world-model family that ships the same architecture twice — an upstream video generator, and a robotics version post-trained with a physics-plausibility reward. Same weight format, same feature tap, same benchmark. The only variable is the post-training.

The physics reward transforms open-loop prediction: the features' ability to forecast the next state improves 50×, from useless to the best zero-shot number we ever measured. Closed-loop control — using those same features to choose actions — does not move at all.¹

Fig. 3 — One post-training step, two metrics, opposite outcomes

upstream vs. physics-RL checkpoint · identical architecture & feature tap · hardened bench

Left: open-loop persistence ratio (prediction error ÷ copy-last-frame error; below 1.0 means the features carry real dynamics; lower is better). Right: closed-loop planning distance after 6 replanning steps (dashed line: random policy). The reward rewrites prediction geometry and leaves control geometry untouched — to four significant figures.

This is not one anomaly. We reproduced the dissociation at four independent layers of the stack: in architecture (a 4B model's tokens hold object position at R² = 0.94, yet every unsupervised readout plans at chance), in distillation, in post-training (Fig. 3), and in action pretraining itself — a 5B policy model trained on 100,000 hours of manipulation has a vision tower indistinguishable from its stock ancestor on every control metric we ran. Wherever we looked, control knowledge stays in the layers nearest the actions and never leaks backward into perception.

The structural read for the market: teams competing on generation quality are not converging on control, however the demo reel looks. The gap is crossed only with action-grounded training — which requires a different, scarcer, and much cheaper-to-collect input, as the next section prices.

§2

Pseudo-labels die three deaths; real actions clear the bar in one

If action data is the scarce input, the tempting shortcut is to manufacture it: run an inverse-dynamics model over ordinary video and train on the inferred actions. We gave the shortcut every chance across three progressively easier versions, and it failed all three, each for a measurable reason.² Continuous pseudo-actions are noise even in-distribution. On real video the pseudo-label anchor doesn't merely fail — it hurts, because the model faithfully memorizes the label noise. And the one channel that is reliable in-distribution (gripper events, r = 0.86) collapses the moment the robot in the video isn't the robot the labeler was trained on.

Then we swapped in genuinely synchronized actions, and the same 1.3M model, same objective, same protocol, cleared the bar immediately — on task variants it had never seen.

Fig. 4 — Held-out prediction ratio: every pseudo-label configuration vs. real actions

ratio = model error ÷ copy-last-frame error · below 1.0 = learned real dynamics · unseen episodes/tasks

Red: pseudo-label configurations on real video — both overshoot the copy-last baseline because the anchor learns the label noise. Grey: no actions at all — pinned to the baseline. Teal: the identical recipe fed real synchronized actions, on held-out task variants; a $30 wrist camera then halves the ratio again. The unlock is not architecture and not scale — it is the provenance of the action signal.

We state it as an observation, not a law, since it rests on the three configurations above: video is abundant and buys little; synchronized action streams are scarce and buy everything. The collection economics favor the small player — a handheld data-collection gripper in the UMI style^· costs a few hundred dollars and produces real action pairs with no robot in the loop; it is the same instrument class behind the largest action-pretraining corpora now being published. The moat is not compute. It is boring, physical, and purchasable.

§3

Two controller families, three levers, one survivor

Closed-loop control is where robot-learning papers go quiet: offline metrics are reported, success rates are not. We ran the loop, then ran the ablations the loop demanded. Two controller families emerged. Behavior cloning — the ACT and diffusion-policy lineage,^· up to and including a fine-tuned 5B VLA — learns a policy from demonstrations. Demonstration tracking, our minimal alternative, learns only a representation and an inverse-dynamics head: it localizes the current state against a reference demonstration in latent space and asks the head for the action that closes the gap.

We then pulled each of the three levers everyone assumes will help — on both families. Click the cells; the pattern is the finding.

Fig. 5 — The lever matrix (click cells to reveal)

closed-loop success · 25 episodes per cell · same demos, same environment throughout

0 of 6 cells revealed — start anywhere.

Reading: better sensing multiplies the tracking family — its failure mode was localization precision, and better pixels fix that directly — while doing nothing for behavior cloning. Correction coverage is what BC theoretically lacks; injecting it changed nothing. Quadrupling demonstrations lifted tracking again and left BC at noise level. In the affordable-data regime, capacity is inversely related to closed-loop success.

The mechanism, once seen, is almost embarrassing: a tracking controller gets its corrections free — re-localizing against the reference every step is error recovery — so uniform scripted demos suffice. A cloned policy must learn recovery from data that contains it, and scripted experts never wobble. The published BC successes rest on human demonstrations, whose natural variance smuggles the corrections in. That variance is a property of the data supply chain, not the algorithm — exactly the kind of fact a build-vs-buy decision should turn on.

§4

Emergence at the edge: the student outruns its teacher

The vision above makes a falsifiable claim: that frontier-model intelligence can be carried onto hardware costing a few hundred dollars without losing what made it intelligent. Our sharpest evidence is a result we did not expect. We distilled a frozen billion-parameter world model^· into a 5M-parameter student — regressing representations, never pixels — and then put both in the control loop. The student did not merely keep up. It beat its own teacher.

Fig. 6 — Teacher vs. student, open loop and closed loop

representation distillation, 3 seeds · closed loop: 6-step receding-horizon planning, final distance to goal

The inversion: the billion-parameter teacher predicts beautifully and plans at chance — its latent distances are dominated by texture, which corrupts the planner's cost surface. The 5M student, forced to compress, keeps the dynamics and drops the texture: distance in its latent space finally means something, and planning works. Precision was not carried down from the teacher. It emerged in the compression.

This is the note's central inversion and the vision's engineering basis. Edge deployment is usually framed as a sacrifice — quantize, prune, accept the loss. In the control loop we measure the opposite sign: the constraint is the regularizer. What the loop needs from a world model is not everything the teacher knows, but the thin slice of it in which distances are decisions — and that slice fits in megabytes. The measured ledger:

Fig. 7 — The edge ledger, measured

same bench, same clips · deployment numbers, not projections

Rows one and two are measured on our bench (encoder latency per 24-frame clip; teacher = frozen ViT-L world model). Row three prices a complete data-to-deployment cell at street prices:³ a hobby-class arm kit, two cameras, a handheld collection gripper, and one used consumer GPU that both trains and serves. The entire 33-experiment program — every number in this note — ran on two such GPUs. There is no cloud line item because there is no cloud.

§5

Two curves worth memorizing

Beneath the headlines sit two scaling behaviors we now treat as planning constants. Within a deployment domain, data volume is honest: error falls smoothly and monotonically as demonstrations accumulate, unsaturated in our range. Across domains, diversity is dishonest: piling on videos from other environments buys one step and then a hard plateau at the do-nothing baseline. Collect deep, not wide — one instrumented workcell outproduces the open internet. (For completeness: a 5B base model, fine-tuned under identical protocol, matched our small models in-domain and paid for itself only on task families never seen in training — insurance against shift you can name, not a default.⁴)

Fig. 8 — What scales (left) and what saturates (right)

left: fraction of in-domain demos, held-out error · right: number of out-of-domain videos, held-out ratio

Left: in-domain data efficiency — 10% → 100% of ~750 clips, error 0.66 → 0.30, still falling at the boundary. Right: cross-domain video count — the first foreign video helps, the next five don't; the curve pins to the copy-last line. Volume compounds only inside the domain that will be deployed.

§6

The stack this audit selects

Thirty-three experiments compress into a single deployment rule. Perception and control are separable investments: cameras and demonstration volume upgrade the edge tier along curves we've measured; frontier models plug in above it, distilled down or adapted few-shot, only where the roadmap names a distribution shift. Every row is priced by an experiment above.

Tier	What runs on the terminal	When it wins	Evidence
Inspect	grade head on frozen features, ≤8% labels	day one, any cell	90% acc at 32% labels
Adapt	1.3M encoder + 120-step few-shot head	new SKU, same cell	0.36 vs 1.13 from scratch, unseen tasks
Control	demo-tracking controller + dual camera	cheap-collection regime	52% and climbing, Fig. 2
Distill	frontier world model → megabyte student	always — the student plans better	Fig. 6: 0.049 vs chance

The uncomfortable summary, and the one we would want stated plainly to us: generation quality is not progress toward control; the binding input is synchronized action data, collectible for hundreds of dollars; and in the loop, the megabyte-scale system doesn't approximate the frontier model — it outperforms it. The cheapest terminal is not where intelligence is compromised. On present evidence, it is where control intelligence actually lives.

METHODS, RAW LEDGERS & NEGATIVE RESULTS: 33-FINDING APPENDIX, ON REQUEST · TOTAL COMPUTE: 2× RTX 3090

Context: the published record we test against

Brohan et al., 2022–23. RT-1 / RT-2: robotics transformers at 35M and 55B parameters. Google DeepMind.

Zhao et al., 2023. Learning fine-grained bimanual manipulation with low-cost hardware (ACT). RSS.

Chi et al., 2023. Diffusion Policy: visuomotor policy learning via action diffusion. RSS.

Chi et al., 2024. Universal Manipulation Interface: in-the-wild robot teaching without robots. RSS.

Kim et al., 2024. OpenVLA: an open 7B vision-language-action model. CoRL.

Black et al., 2024. π0: a vision-language-action flow model for general robot control. Physical Intelligence.

NVIDIA, 2025. GR00T N1: an open foundation model for humanoid robots (2B).

Assran et al., 2025. V-JEPA 2: self-supervised video models enabling understanding, prediction and planning. Meta AI.

Zhou et al., 2025. DINO-WM: world models on pre-trained visual features enable zero-shot planning. ICML.

Xiaomi Robotics, 2026. Breaking the data barrier: scaling robot policy models with embodiment-free pre-training (XR-1, 5B).

0. Model sizes are public release figures (see references); points are placed at announcement dates. "Edge-feasible" in the hover annotations means the policy loop runs at control rate on a ≤$1k embedded or consumer device without quantization heroics — our judgment, stated so it can be argued with.

1. Open loop: persistence ratio on a held-out SKU (prediction MSE ÷ copy-last MSE), mean-pooled features, 3 seeds. Closed loop: final distance to goal under 6-step receding-horizon planning with per-step replanning; random-policy reference 0.416. The two checkpoints differ only in weights; architecture, tap layer and protocol are byte-identical.

2. Pseudo-labels from a 4B inverse-dynamics pipeline at its official sampler settings, validated against ground truth on its own training distribution before use. The three deaths: continuous dims in-distribution (per-step r≈0); continuous dims on real video (anchor memorizes noise: train ratio 0.16, eval 1.44); event channel out-of-distribution (train 0.09–0.28, eval 2.4–3.9).

3. Approximate street prices, 2024–26: hobby-class 6-DoF arm kit ≈ $130 (SO-101 class); two USB global-shutter cameras ≈ $60; UMI-style handheld gripper build ≈ $400; used RTX 3090 ≈ $700. Prices vary; the point survives a 2× error.

4. Same-split comparisons, two robots: in-domain 0.646 (3M scratch) vs 0.676 (5B fine-tuned); cosmetic shift 0.184 vs 0.408; semantic shift 10.30 vs 9.69 (zero-action baseline 11.35). Full protocol in the appendix ledger.

· Chi et al. 2024, references. · Zhao et al. 2023; Chi et al. 2023. · ViT-L world model of the V-JEPA family (references), frozen; distillation regresses per-tubelet representations only.