Molmo2 Codec V6 — Zero-Shot Stage-Boundary Probe Log

Native RGB

61.5%

RGB81 · 123 / 200 correct

Stage-1 reconstruction

61.5%

Hφ81 · 123 / 200 correct

Deployable codec

58.5%

real P16 · 117 / 200 correct

Zero-shot baseline: before any Stage-2 LLM tuning, real P16 is 3.0 accuracy points below RGB81 while using 26.4% of its mean visual-token count.

Do not label this as Stage-2 performance: the LLM is the original Molmo2-4B-SFT model.
Do not mix this result with the earlier native probe: the sample, token budget, and Hφ/RGB output widths differ.
Accuracy parity is aggregate only: RGB81 and Hφ81 each score 123/200 but agree on correctness for 170/200 examples.

2 · Exact evaluation contract

Field	Recorded value
Dataset	MLVU multiple choice · validation · stratified n=200
LLM	Molmo2-4B-SFT · no Stage-2 checkpoint loaded
Stage-1 artifact	codec_stage1_full_v6_delta/ptokenizer_step17000.pt
Sampler	full 2-fps timeline → adaptive GOPs → complete-GOP selection
Codec visual budget	1,377 tokens
CLI max_frames	17; codec GOP selection still fits 31–68 compressed frames
Frame control	all three modes use identical selected frame IDs
Raw shard artifacts	stage_boundary_probe_17f_n200_job215_s0..s7.json

3 · Paired outcome log

Paired correctness comparison	A correct / B wrong	A wrong / B correct	Same correctness
RGB81 vs. Hφ81	15	15	170
RGB81 vs. real P16	19	13	168
Hφ81 vs. real P16	9	3	188

Hφ reconstructs a useful native-space representation: its aggregate accuracy matches RGB81 on this sample.
Real P is already consumable zero-shot: it preserves 117/123 of the aggregate correct-count baseline at substantially lower token cost.
n=200 is diagnostic, not final: use this page as the Stage-2 step-0 anchor for checkpoint curves.

4 · Why the earlier 67.5 / 69.5 / 68.5 result is different

Probe	Sample	Codec budget	Mean selected frames	RGB	Hφ	real P
This stage-boundary probe	stratified n=200	1,377	56.59	RGB81: 61.5%	Hφ81: 61.5%	P16: 58.5%
Earlier native probe	dataset rows 0–199	2,592	116.03	RGB16: 68.5%	Hφ16: 69.5%	P16: 67.5%

Both are valid logs, but they answer different questions: the earlier probe compares three 16-token P-frame controls; this probe compares deployable P16 against full-width RGB81/Hφ81.
No cross-row delta is causal: sample composition, visual budget, selected-frame count, and representation width all change together.

1 · Result