Learned P-tokenizer vs. Pruning-16 vs. Compression-9 · 2026-08-26
Molmo2P-framePruning-16Stage-2decision record
Decision
Selected: Pruning-16. Keep 16 real Molmo connector tokens per P-frame.
Stopped: Compression-9. It is smaller, but its extra pooling bottleneck trails Pruning-16 in the controlled step100 gate.
Legacy: learned P-tokenizer. Compact and motion-aware, but requires a separate alignment stage and creates the largest feature-distribution risk.
The key distinction: Pruning-16 selects native visual tokens; the other two routes synthesize a new compressed representation.
One frame, three operations
Dense Molmo produces a 9×9 connector grid: 81 visual tokens.
Pruning-16 scores the 81 cells with local residual and motion magnitude, keeps the top 16, then restores raster order.
Compression-9 merges every 3×3 group of connector cells into one output token.
Pruning preserves selected native cells and their spatial order; Compression-9 replaces each 3×3 region with a pooled token.
Efficiency + controlled step100 evidence
Controlled comparison: identical MLVU stratified n=200 index, validation split, 8K visual budget, and multiple-choice log-prob scoring. Pruning-16 and Compression-9 each use their own matched step100 checkpoint.
Pruning-16 uses 44% more P tokens than Compression-9 (16 vs. 9) and gains 3.0 accuracy points in the first matched evaluation.
Why Pruning-16 is the safer route
Property
Learned P-tokenizer
Pruning-16
Compression-9
P tokens / frame
16
16
9
Token identity
New learned representation
Selected native Molmo tokens
New pooled representation
Needs Stage-1 codec alignment
Yes
No
No separate Stage-1
Uses motion / residual
As model input
As selection score
No spatial selection
Main risk
Feature-distribution mismatch
May discard useful static cells
Over-compression / pooled-token shift
Current status
Legacy reference
Formal run
Stopped after gate
Formal run contract
8× H200, BF16, FSDP2, global batch 128.
Frozen shared Molmo ViT; independent P-connector and LLM are trainable.
RGB HEX K4 GOPs; full 32K sequence with a 16K visual cap.
Checkpoint step400 is verified; the run resumes toward step1000 with saves every 100 steps.
The current run removes Compression-9 entirely and keeps a 50/50 codec/native training mixture.
Reading the result correctly
The step100 result is a route-selection gate, not the final model claim.
The earlier learned P-tokenizer result belongs to a historical recipe, so it is not inserted into the controlled two-way bar chart.
The final decision will use the verified step1000 checkpoint and the same fixed evaluation index.
Evidence: formal native-P contracts, the fixed MLVU n=200 paired summary, and the verified Pruning-16 step400 checkpoint in the Molmo2 Codec workspace.
