Vision → SONIC Humanoid Ball-Catch

Driving frozen SONIC from vision via its latent token is the right interface. Our failures were training-recipe, not interface — missing proprio strong-fusion, on-policy distillation, and a ball-localization head.

Goal: a vision policy drives a G1 to walk to + catch a ball; frozen SONIC (enc → FSQ token(2×32) → dec → action) is the low-level controller.

1 · Overfit one trajectory first (method)

Method: before generalizing, overfit a single ground-truth trajectory closed-loop with the token-route policy below. It reproduces the full walk → catch, upright — so SONIC + the pipeline can do the task; generalization is the open problem, not capability.

1 · videos: ground truth vs overfit policy

Ground truth — expert demo (SONIC tracking the BM plan).

Our overfit policy (β=0): reproduces walk → catch closed-loop.

2 · Pivot → reference-motion route (doesn't work well)

To generalize we tried having the model output a reference motion for frozen SONIC to track. It doesn't work well. The model overfits the reference but underfits even one episode (joint MAE 0.08 vs target jitter 0.008): the vision-centric arch dilutes proprio to one token, so it can't pin body pose → a smoothed, laggy gait that doesn't truly walk or reach.

3 · Token route + generalization (BC + DAgger)

BC generalizes teacher-forced but falls closed-loop. DAgger r1: 0/6 falls on fresh spawns, walks to the ball — but never catches (arms stay at walk level). Closed-loop eval is high-variance: same ckpt + seed → upright once, fell once.

r1, good spawn: walks to 0.48 m, upright.

r1, hard spawn: heading veers, under-shoots ~4 m.

4 · New model arch — VIRAL vs GRAIL

Both drive SONIC from vision but differ in how. VIRAL: ResNet vision + proprio (concat + history cross-attention) + an obj_pred head → command/token, trained by on-policy DAgger (teacher labels student-visited states). GRAIL: keep SONIC frozen, take E(reference) as the base token and learn only a small residual Δz (λ=0.1, ℓ₂, re-quantized) via RL, then distill to vision→token.

Our alignment with GRAIL's latent control: same interface — token / frozen SONIC / FSQ / teacher = E(reference) / distill-to-vision. We deliberately simplify for catching: (1) no learned residual Δz — GRAIL needs it for dexterous grasp beyond SONIC, but our catch is within SONIC's tracking repertoire, so E(BM ref) alone suffices; (2) ball info via an obj_pred aux head instead of GRAIL's privileged object input; (3) supervised / on-policy DAgger instead of RL (no residual to learn). If finer hand control is ever needed, the residual can be added back.

5 · Decision

ViralSonicPolicy: VIRAL-style ResNet + proprio (concat + history cross-attn) + ball obj_pred head → SONIC token → frozen FSQ + decoder; trained by on-policy DAgger (teacher = SONIC-encoder on the re-anchored BM reference). Sanity-overfit already fits the token at 13.7 M params / 1.6 GB (vs WildDet3D 1.18 B).