AdaCodec vs. LLaVA-OneVision-2
A visual comparison for the Molmo2 codec project · 2026-06-21
The 20-second answer
AdaCodec
Compresses every frame into learned change tokens
Keeps the timeline dense. Motion + residual become 16 latent P-tokens per predictive frame.
VS
LLaVA-OneVision-2
Uses codec signals to choose which RGB patches to keep
Keeps exact visual detail. Important patches win a shared budget; other frames may disappear.
Core trade-off: AdaCodec protects time coverage; OV-2 protects spatial detail.
1 · The two pipelines
AdaCodec
NEW LATENT LANGUAGEDecoded RGBPrevious + current frame
→
Motion + residualCustom block matching
→
P-tokenizerSeparate ViT branch
Output: 16 learned P-tokens / frame
The codec signal itself becomes the model input.
LLaVA-OneVision-2
SPARSE RGBVideo bitstreamBit-cost / MV / residual
→
Patch rankingTop 2×2 RGB blocks
→
RGB canvasesShared vision encoder
Output: selected real RGB patch tokens
Codec signals choose the input; they are not the final input.
2 · Where the token budget goes
AdaCodecEvery sampled moment survives
256
16
16
16
16
16
16
16
OV-2Frames compete for shared RGB space
full
RGB
0
RGB
RGB
0
RGB
RGB
Bar height illustrates relative token allocation, not measured values for OV-2.
AdaCodec wins when continuity mattersStreaming, monitoring, action order, persistent state.
OV-2 wins when exact pixels matterOCR, small objects, event localization, spatial evidence.
3 · Which method fits which task?
More temporal continuity →
Live + temporal
AdaCodec · camera assistant
Hybrid · trajectory tracking
Live + spatial
Hybrid · robot manipulation
Sparse RGB · text/details
Offline + temporal
AdaCodec · long monitoring
OV-2 · event retrieval
Offline + spatial
OV-2 · OCR / grounding
OV-2 · fine appearance
More exact spatial detail →
AdaCodec favored
OV-2 favored
Hybrid favored
4 · Recommended Molmo2 design
RGB anchorGlobal identity, scene layout, text
+
Selected RGB patchesOV-2 idea: preserve high-detail events
+
Tiny delta tokensAdaCodec idea: guarantee state for every time step
Molmo2 hybrid: exact pixels where needed + a cheap temporal heartbeat everywhere.
5 · Minimal experiment plan
01
Full RGB baselineLock Molmo2, data, timeline, and token budget.02
Sparse RGB onlyMeasure OV-2-style selection without latent deltas.03
Sparse RGB + deltasTest whether temporal heartbeat recovers missed motion.Keep these caveats visible
Not an apples-to-apples benchmarkDifferent backbones, training data, token accounting, and latency protocols.
AdaCodec is not yet reproducibleIndependent code and checkpoints were not public on 2026-06-21.
OV-2 depends on file encodingCodec, bitrate, QP, and GOP changes can alter which patches are selected.
Sources and technical notes
LLaVA-OneVision-2 report · repository · codec-video-prep
AdaCodec reports 256 tokens for a 512×512 I-frame and 16 P-tokens per P-frame. The OV-2 bars above show its group-level competitive allocation conceptually; they are not a published per-frame token trace.
Visual research note · project: molmo2_codec · public sources inspected 2026-06-21