Where long horizons break policies
Visuomotor policies are good at short, well-defined skills and degrade over long horizons. Errors compound from one sub-task into the next, and a policy trained to imitate successful trajectories has no way to represent the decision to stop, back up and try again. Covering a whole long-horizon task with demonstrations, including every way it can go wrong partway through, is not a data budget anyone can afford.
Specialists, and a judge above them
- Individual short-horizon specialist policies trained with ACT: 60 demonstrations each with location randomization, 70k training steps at batch size 64.
- An LLM judge on top that reads the scene, chooses which specialist to invoke next, and decides when the current one has finished or failed. The slow deliberative layer over the fast learned skills.
- The top level's action space is the set of specialists, not raw robot actions. Long-horizon structure becomes something the system reasons over explicitly, rather than something one policy has to absorb from data.
- Demonstrated end to end on cup stacking, about 80% success across 20 consecutive rollouts, built over a single hackathon weekend.
What the hierarchy bought
- Decomposition is what made the data budget work. Each specialist only has to cover a few seconds of behavior, so 60 demonstrations per skill was enough; the long-horizon burden moved out of the policy and onto the LLM.
- Location randomization in those 60 demonstrations is what let each specialist tolerate whatever state the previous one left behind. Without it, the handoff between skills is the thing that breaks.
- Making failure a first-class transition. The LLM can decide a skill has failed and re-invoke it, which is behavior a monolithic policy has no way to express at all.
