Two-layer part-quality stack for the 3D editing data engine · 2026-07-22
3d-editingpart-segmentationVLMdata-engine
TL;DR
The correct edit unit is neither a raw P3-SAM part nor the whole object — it is a
co-move group built by two complementary layers: geometry (for what the VLM can't SEE) + VLM
(for what geometry can't KNOW).
Downstream: part-removal 9/10 → 12/12; semantic "add a hat/wings/tail" pairs
28/30 (93%) — with no X-Part, no GPU, real geometry.
1 · Problem — flat segmentation vs nested semantic parts
P3-SAM gives a flat set of parts; real semantic parts are nested (a head contains
eyes/teeth; a car body contains windows).
"Move the head" on a flat part id moves only the outer skull shell — the eyes stay in place,
floating in the old position. Same failure hides behind "remove the arm" and the E10 chains.
~8% of pilot assets have hidden interior parts (max 12 contained parts on one asset).
Move the head. OLD (middle) moves the single P3-SAM part → eyes/teeth left behind (horror). NEW (right) moves the co-move group → the whole head travels.
2 · Geometry layer — three moves (handles what the VLM can't see)
Winding-number containment: a part inside another (eyes in a closed head) — generalized
winding number of Q's samples w.r.t. P's face soup > 0.5. Robust to open shells (igl).
Nested-shell clustering: double walls / a jacket hugging a torso — merge when contact
fraction ≥ 0.35 or bbox-IoU ≥ 0.50 (relative to the smaller).
Tangent-continuity seam merge: two halves of ONE surface split by the segmenter meet along
a rim where normals continue smoothly (median angle < 22°); a real joint meets at a concave crease.
Edit unit = cluster + all descendants. Per-asset, CPU, seconds. Stored as
part_tree.json {tree, cluster, group}.
After all three moves: the human head (inner+outer wall + eyeballs) moves as one; the cat's face/mouth/teeth all follow. No stray parts.10 assets, move-a-part. [original | OLD single-part | NEW co-move group]. E.g. row 6: the whole pink-haired rider (13-part group) moves off the dragon; row 8: the chick's comb follows the head.
3 · VLM layer — group + name in one call (handles what geometry can't know)
Geometry cannot tell "rider vs dragon" or "sword vs hand" — they're physically touching but
semantically distinct. That needs vision.
Color-code the geometry clusters with flat colors, render 3 views + a legend, and make
ONE Qwen3.6-27B call that groups colors into named semantic parts
(head = red+brown+cyan+magenta).
One call does merging and naming — replaces per-pair part naming, and the group name is the
instruction word ("move the comb").
The colored input the VLM sees: each geometry cluster a distinct flat color + legend. It reasons over solid colored regions, not transparent point scatters.Move a semantic part. [before | geometry co-move group | VLM semantic group]. Row 2 (head): geometry alone moves only the eyeballs (1 pid); the VLM group unifies 4 clusters → the whole head moves, hands & base correctly left.
4 · Two-layer division + downstream gains
Geometry handles what the VLM can't SEE: hidden interior parts (eyeballs inside a closed
head), double walls, split-surface seams — invisible in any render.
VLM handles what geometry can't KNOW: rider-vs-dragon, sword-vs-hand — physically adjacent
but semantically separate.
Each covers the other's blind spot; neither alone is sufficient.
Instruction wording, paint-range, and 3D mask all align by construction — the group name IS the
instruction.
Task
Before
After (semantic units)
E3 remove part
9/10
12/12
E2 add part (semantic reverse: "add a hat/wings/tail")
6/10 (X-Part chain)
28/30 = 93%
"Add a part" needs no X-Part: remove a real VLM-named accessory (hat/wings/tail),
reverse the pair. Geometry & texture are real, placement perfect, $0, no GPU.
30 semantic add-part pairs [before: without | after: with], green = dual-blind judge pass. 28/30. Natural instructions: add a hat / staff / wings / tail / helmet / boots.
Method report · part-quality stack for the WilD3DGen editing data engine · single pilot (200 assets)
