← Back to project

Molmo2 Codec — Native Pruning-16 Step 1000

Training-identical native visual path · RGB-HEX K4 · frozen-ID paired evaluation · 2026-08-29
Pruning-16Stage 2RGB HEXK4MLVUrelease

1 · Release result

MLVU Dense
79.0%
158 / 200
MLVU Codec
80.0%
160 / 200 · +1.0 pt
Six-suite Codec mean
69.50%
Dense 71.67% · -2.17 pt
P-frame compression
81 → 16
5.06× fewer visual tokens per P frame
Release verdict: Pruning-16 is a functional native-space codec path. It reaches 80.0% on MLVU and 69.50% over the six-task suite. The aggregate six-task gap to Dense is -2.17 points; individual regressions remain visible below.
Frozen-ID paired suite · n=200 per benchmark Bars: step1000 Dense / Codec · violet dot: step920 Codec 020406080100 Video-MME68.567.5 (-1.0)LVBench50.545.0 (-5.5)MVBench76.073.5 (-2.5)TempCompass69.568.0 (-1.5)PerceptionTest82.079.5 (-2.5)NExT-QA83.583.5 (+0.0) Dense Pruning-16 step1000 Pruning-16 step920

2 · What changed

One frozen native visual space; compression happens by cell selection P-frame RGBselected source frame no RGB pixel cache Frozen Molmo ViT729 patch features 81 native 3×3 cells robust motion + residual saliency keep top 16 · restore raster order P connector copyindependent · Stage-2 trained 16tokens I-frame control path stays native frozen Molmo ViT + native connector → 81 tokens No Stage-1 P-tokenizer is loaded. The complete Stage-2 checkpoint contains the trained independent P connector.

3 · Exact paired benchmark table

BenchmarkDensePruning-16 step1000Codec − DensePruning-16 step920
Video-MME68.567.5-1.068.0
LVBench50.545.0-5.544.0
MVBench76.073.5-2.573.5
TempCompass69.568.0-1.568.0
PerceptionTest82.079.5-2.580.0
NExT-QA83.583.5+0.084.5
Six-task mean71.6769.50-2.1769.67
Five-task mean, excluding LVBench75.9074.40-1.50

4 · Training and artifact contract

Resume920 → step1000 continued to optimize cleanly Cross-entropy falls 5.15% over the final 60 steps; final train accuracy is 79.88%. 0.72409400.71729600.70589800.68671000 optimizer step
FieldRelease value
Checkpointpruning16/step1000_consolidated.pt
Checkpoint audit8 rank states · 128 tensor shards · optimizer verified · zero NaN tensors
Final train metricsCE 0.6867 · accuracy 79.88% · dynamic MFU 17.76%
Visual audit at step100013,551 I frames · 49,702 P frames · 2,279,718 visual-token rows
Input contractRGB HEX · gamma 19,439,676 · target/max K4 · 2 FPS · full-span
Release repositoriesGitHub weikaih04/molmo2-codec · HF weikaih/molmo2-codec-pruning16-step1000
Source: immutable checkpoint audit, eight MLVU shards, and six frozen-ID lmms-eval paired summaries.