← Distortion LabSix axes take the fix. One does not.
LIBERO-Plus perturbs a saturated manipulation benchmark along seven axes — background, camera, language, layout, lighting, sensor noise, and the arm's own starting joint configuration — and asks a frozen policy to carry on. Six of those axes respond to the obvious remedy. Retrain on perturbed demonstrations and the scores come back.
The seventh does not. Fine-tuning the leaderboard holder on 20 000 generated trajectories buys 37.2 points on camera perturbation and 8.6 on robot initial states, from the same data. Measured against standard OpenVLA‑OFT rather than the multi-task variant, those trajectories buy nothing at all.
The stepped strip here is a real test strip: seven patches, each exposed to the published score of the current leaderboard holder on that axis. Select a patch to read it.
The axis is not a perception test.
It is a kinematics test.
The perturbation moves joint angles, by 0.1 to 0.5 radians in the infinity norm. Every published intervention we know of treats the consequence as a perceptual or semantic discrepancy and acts in Cartesian end-effector space. We measured how much of the joint displacement actually survives into each Cartesian channel.
A median joint displacement of 0.29 rad produces a median gripper displacement of 0.039 m and a median gripper rotation of 0.432 rad. Four centimetres is inside what the policy tolerates. A quarter-turn of the wrist is not.
That asymmetry decides intervention design, and the method's own ablation confirms it from a second direction: over 120 shared variants, homing on position alone reaches 37.5% [29.4, 46.4] and homing on all six degrees of freedom reaches 86.7% [79.4, 91.6]. An intervention that moves the gripper without rotating it is addressing the small half of the disturbance.
Correlation with joint distance:
position r = +0.64 · orientation r = +0.85
- Gripper displacement
- 0.039 m
- Gripper rotation
- 0.432 rad 24.8°
the small half
the half that costs the episode
Put the arm back where the demonstrations start.
Classical pre-image analysis says that executing a task does not require a controller valid over the whole configuration manifold. It requires entering a region where the nominal controller already holds. On this axis, narrow demonstration support defines that region, and the arm simply begins outside it.
So for the first W steps we add a homing term to the policy's own action, mapping the joint error through the forward Jacobians the operational-space controller already computes, and then release. The policy is frozen. The gripper channel is untouched — it decides what the task does, not where the arm is.
The forward Jacobian, not its inverse: the joint error is mapped into the task-space command the controller already accepts. One consequence is structural — error in the Jacobian's null space is not addressed by construction — and we measured whether that matters. It does not: failures carry less null-space error (0.157) than successes (0.201).
- Window
- 12steps
- Gain
- 20
- Learned parameters
- 0
- Added compute
- 16×7 matmul / step
At this gain the command is often large: the median correction is 0.254 against an action range of ±1, but the p90 is 2.49, so 17.4% of corrected channel-steps hit the clip. The clipping is not spread evenly — it is 2% per translation channel against 16%, 42% and 40% in roll, pitch and yaw. The orientation half of the controller frequently operates saturated rather than proportionally, which is a third independent sighting of the same asymmetry.
What it is not
It is not a teleport. Writing joint positions directly destroys episode dynamics — it takes the same tasks from 73% to 7%. The arm is driven there through the same action interface the policy uses, subject to the same controller and the same physics.
It is also not free of assumptions. It needs q⋆, the canonical configuration, which we estimate from ten variants of a perturbation column that leaves the physics untouched. That estimate is the method's one parameter, and the paper states what an error in it costs.
Twenty-four steps of return.
Then the frozen policy has it back.
The column, under the benchmark's own protocol.
Robot Initial States column. OpenVLA-OFT+ (7B), seed 777 tuning split (240 variants) and full 1,550 column runs, one episode per variant. Paired execution — all arms evaluated under identical simulator states and perturbation seeds.
| intervention arm | n | baseline | treated | fixed / broke | p (McNemar) |
|---|---|---|---|---|---|
| Jacobian Return (home_first, W=24) | 240 | 30.4%[24.9, 36.5] | 97.5%[94.7, 98.8] | 166 / 5 | p = 0.004 |
| Learned Return (home_learned, 135k MLP) | 240 | 30.4%[24.9, 36.5] | 96.7%[93.6, 98.3] | 163 / 4 | p < 10⁻³³ |
| Gripper-Pose Return (home_eef, no joints) | 240 | 30.4%[24.9, 36.5] | 95.0%[91.5, 97.1] | 161 / 6 | p = 0.146 |
| Concurrent Steering (earlier formulation, W=12) | 1 550 | 23.5%[21.5, 25.7] | 83.5%[81.6, 85.2] | 950 / 21 | χ² = 887 |
Return-then-release lifts policy success to 97.5% — matching the policy's own unperturbed baseline on clean vanilla LIBERO (95.5%). Muting the arm commands for 24 steps costs nothing in execution budget (median successful episode: 139 steps vs. 145 untreated).
Crucially, homing is completely free on clean starts: evaluating on 1 000 vanilla unperturbed episodes shows +0.5 pp [−1.1, +2.1] for Jacobian return and +1.7 pp for the learned module. No out-of-distribution detector is needed: the return routine runs safely on every episode without degrading nominal performance.
For context, and measured by other groups on other harnesses: standard OpenVLA‑OFT publishes 31.9% on this column, official leaderboard top is 30.3%, and best reported model is 46.2%. We do not subtract across that boundary — the lab’s sixth order says why.
Per suite, over the full column.
| suite | n | baseline | corrected | fixed / broke |
|---|---|---|---|---|
| libero_10 | 393 | 33.6% | 92.1% | 236 / 6 |
| libero_goal | 409 | 18.6% | 72.1% | 228 / 9 |
| libero_object | 398 | 20.1% | 92.0% | 288 / 2 |
| libero_spatial | 350 | 22.0% | 77.4% | 198 / 4 |
Can the policy learn to do it itself?
Cancelling a perturbation from outside is not the same as being robust to it, and we take that objection seriously enough to have run it. A 590 000-parameter residual module, sitting beside an otherwise untouched 7B policy, reaches 49.0% on the 934 variants it never trained on.
On one identical set of 100 variants the three arms read 32.0% frozen, 54.0% with the module, and 88.0% with the analytic controller. The module recovers about two fifths of the gap and stops.
What separates them is sensing rate, not capacity. The analytic controller re-reads the joint error before every action. A chunked policy reads it once per eight steps, and across that chunk the correction's predictability falls from R² = 0.897 at the first position to 0.410 at the eighth. Handicapping the analytic controller to that same observation rate drops it to 21.0% — below the untreated policy.
Everything we tried first, and why it failed.
Before the kinematic method there were seven policy-side interventions. Two of them are significantly negative. They are on the line because a lab that only shows its successful prints is not reporting, it is advertising.
Three mechanisms, not three accidents
An affine on visual tokens cannot represent the fix. We fitted one analytically, by gradient descent, and finally as an oracle directly on the test distribution. The oracle repairs six episodes and breaks six. An intervention whose oracle is worthless is not under-fitted — it is the wrong instrument.
Capacity makes it worse. The 111k-parameter per-block variant, a 25× expansion, broke 21 episodes while repairing 7.
The most instructive failure
We fine-tuned the 7B policy toward its own recorded actions plus the correction — a target on which 78% of supervised steps prescribe no change at all. The loss minimised cleanly. Closed-loop, it then succeeded on none of the 80 holdout tasks, against 22.0% frozen.
The cause: OpenVLA-OFT trains teacher-forced and infers by parallel chunk decoding, so "the model's own output" recorded at inference is not what the training forward pass produces. The anchor was not an anchor. The rule we now keep: before trusting an "own output plus correction" target, measure the loss on frozen weights and require it to equal the correction's own magnitude, not more.
Before any claim, the harness had to earn it.
Nothing on this page is a leaderboard submission. Every figure in it is held-out-task measurement under our own harness, and the harness had to earn that role first: running the leaderboard holder unmodified over a stratified 3 300-episode sample puts six of seven columns within 4 p.p. of the published table, and reproduces the column that matters here to 0.5 p.p.
The seventh column is the interesting one. We read Sensor Noise 6.3 points high, reproducibly, and the cause is established — it is a defect in what that axis actually perturbs, not a policy that got lucky. We report it as a failure of reproduction rather than a win.
- One episode per variant, arms paired over identical variants, seeds and perturbations.
- Wilson intervals on every proportion; McNemar on every paired difference.
- Held-out axes declared in writing, with a timestamp, before the run that uses them.
- Rates replicate and individual outcomes do not: two runs on one GPU agree 120 / 120, across GPU architectures 232 / 240 — and both still report the same rate to two decimals.
- Per-episode logs released with the paper, so the columns can be recomputed without running the model.
Reproduction gate. OpenVLA-OFT+ unmodified, our harness, one episode per variant, episode-weighted average — run before any number on this page was measured.
| column | n | ours | published | Δ |
|---|---|---|---|---|
| background | 354 | 96.6% | 93.9 | +2.7 |
| camera | 526 | 94.5% | 92.8 | +1.7 |
| language | 505 | 86.7% | 85.8 | +0.9 |
| layout | 502 | 74.7% | 77.6 | −2.9 |
| light | 376 | 93.1% | 94.9 | −1.8 |
| noise | 527 | 95.6% | 89.3 | +6.3 |
| robot | 510 | 30.8% | 30.3 | +0.5 |
| average | 3 300 | 80.7% | 79.6 | +1.1 |
Robot Initial State Is Not a Robustness Problem: Return, Then Release, Around a Frozen VLA Policy
The Robot Initial States axis resists the remedy that works on the other six. We show the axis does not test visual or semantic robustness but a kinematic one — the arm begins outside the narrow band of joint configurations the demonstrations cover — and that returning it there recovers most of the column.
Pausing the policy for 24 steps while a training-free Jacobian controller returns the arm to its canonical configuration (return-then-release) lifts OpenVLA‑OFT+ from 30.4% to 97.5% [94.7, 98.8] over tuning-split variants (p = 0.004, 166 fixed / 5 broken) and is free on clean vanilla LIBERO (+0.5 pp). A 135k-parameter network imitating that return matches it at 96.7%, and a gripper-pose return reaches 95.0% without joint encoders. We also report seven policy-side interventions that failed, and why earlier learned residuals stopped at half.
Contributions
Return, then release
Dropping the policy's arm commands for 24 steps while returning the arm to home pose beats concurrent steering (97.5% vs. 91.2%), reaching unperturbed vanilla levels at zero cost on clean starts.
A learned return, under the full protocol
A 135k-parameter MLP imitating the controller from joint angles reaches 96.7% on the column (R² = 0.996) with zero benchmark variants in training.
Gripper-pose return
Homing driven solely by gripper pose error (zero joint encoders or Jacobians) reaches 95.0% and is indistinguishable from the Jacobian return up to 0.3 rad.
What the column measures
The benchmark's training set leaves the axis out, a 24-step homing routine closes the column, and published scores on this axis vary by up to 35 points across harnesses.
Why learned residuals stopped at half
Action chunking and open-loop execution compound errors; replacing actions in a fixed window eliminates the compound drift.
What this paper does not claim
Four open items, each stated in the paper as a limitation rather than papered over.
- No leaderboard entry. An aggregate score of 83.2% was computed and then withdrawn: 10.5% of the benchmark had been in the module's training set.
- The module is not exactly inert. Paired over 2 634 clean episodes it costs −0.57 p.p. [−1.15, +0.01]. The trade is favourable by roughly thirty to one, but it is a trade.
- The training-free arm has one parameter after all — the reference pose. An error of 0.10 rad in it puts the corrected policy below the untreated one.
- Our OpenVLA-OFT baseline reads 23.5% against a published 31.9%. Determinism is ruled out and the harness reproduces six other columns. The cause is unknown and we say so.
The preprint is not yet posted. When it is, the per-episode logs go with it — the lab’s fifth order. Until then, write to lab.distortion@gmail.com and ask.