WilD3DGen · Generative editing engine: Nano3D-v2 variant on TRELLIS.2
first FlowEdit on TRELLIS.2 · coordinate-frame findings · 11-asset P-vs-G A/B · 2026-08-11
editingdata-engineFlowEditTRELLIS.2A/B
0 · TL;DR
First working FlowEdit on TRELLIS.2 (no published precedent) — null-invariant + part-mask anchoring validated; P3-SAM part masks replace Buffalo's learned 3D-bbox planner, directly targeting their admitted box-interior-noise problem.
11-asset A/B: procedural route removes 10/11 (but leaves stumps); pure FlowEdit route only 1/11 clean — deletion signal from 2D conds is too weak, parts get regrown. Fix identified: HYBRID — procedurally clear part voxels, generate only the healing.
Two silent data landmines caught pre-production: independent per-mesh normalization (0% voxel retention on geometry edits → 100% with a shared union frame) and a two-loader provenance split (Blender-evaluated pose vs trimesh bind pose; caches match current GLBs on only 8/199 assets) → single-loader self-encoding is now the standard.
Our substitutions: P3-SAM part voxel masks instead of the learned bbox (we know the edit region exactly — they must predict it); QIE-Lightning 8-step instead of Qwen-Image; exact silhouette alpha from the procedural after-render instead of pixel-diff masks; TRELLIS.2 decode instead of closed-source LATTICE/NaTex.
Everything lives in ONE trimesh load: mesh, parts, self-encoded latents, cond renders.
Runtime per asset: encode 3-5 s · QIE 8-step ~5 s · SS FlowEdit ~8 s · SLAT stages ~40 s.
Single-loader world: the same trimesh load feeds masks, latents and renders — provenance mismatches are impossible by construction.
TRELLIS normalizes every mesh to [-0.5,0.5] independently; for geometry edits the before/after frames differ → unedited geometry changes voxel address.
Measured (res 64 & 512): unedited-voxel retention 0% for delete/scale/move; 100% under a shared union-bbox frame. Literature: only EditVerse3D names this (joint normalization; 'fails to converge' without) — no one quantifies it.
Frame choice is NOT free: source-frame hard-fails on additions, after-frame on deletions (range assert) — union bbox is the only always-safe choice and degrades to the right frame automatically.
Material edits (E1a/E1b) freeze geometry → identical frames → immune; this also explains their pilot dominance (29/30) vs geometry tasks.
Left: the 0%→100% retention flip. Right: one rule — bbox(source ∪ after), stored per pair, never recomputed downstream.
3 · Provenance landmine: same file, two loaders, two geometries
Legacy latent caches were encoded via the Blender pipeline (armature EVALUATED, per-asset orientation); the edit engine loads GLBs with trimesh (skeleton ignored → bind pose).
Gate (mesh voxelization vs SS-decoded occupancy, IoU≥0.85): only 8/199 pilot assets agree. Failures split into pure-orientation (fixable by rotation) and pose differences (animated rigs — unfixable by any axis permutation).
Not file corruption: dates disprove re-download; visual proof below. Same family as the earlier silent 13% / 90°-texture misalignment in EditV1.
Consequences: main generation training is SAFE (renders↔latents share the Blender world, internal IoU 0.999). Any pipeline mixing worlds is not → fixed by self-encoding (3-5 s/asset, in-line, nothing persisted).
The same .glb: crouched (Blender, evaluated skeleton) vs lying flat (trimesh, bind pose). The cache decodes to the Blender world — mixing worlds poisons any voxel-aligned pipeline silently.
4 · FlowEdit on TRELLIS.2 — validated core
All published FlowEdit editing works run on TRELLIS-1; TRELLIS.2 had none. Implemented on the official 4B flows (SS dense 16³ latent + sparse shape/tex SLATs).
T1 null-invariant: same condition both branches → decoded occupancy bit-identical (latent 1e-2 residue = fp roundoff of (x+z)−x amplified by the 1.3B net — formula-inherent).
T2 masking: frozen region diff exactly 0; free region edits (IoU 0.942 cross-view).
Gotchas fixed en route: flex_gemm autotuner passes warmup=None into newer triton (patch BOTH triton.testing and triton.runtime.autotuner — the latter binds by name); np.float64 × SparseTensor explodes numpy broadcasting (t-sequence must be Python floats); low_vram pipelines park models on CPU.
5 · End-to-end single asset
Full chain: auto anchor view (geometric part-visibility proxy for Buffalo's VLM salience) → QIE edit → SS FlowEdit → anchored shape/tex → GLB → renders.
P vs G trade-off in one image: P is exact but leaves an open stump (interior exposed, renders black/white); G removes AND closes the neck plausibly, frozen claws/body intact.
'Remove the head': ③ P exact + stump vs ④ G closed. This asset is the clean G success — the batch shows it is not yet representative.
6 · Batch A/B (11 assets): the honest result
P procedural: 10/11 clean removals (stump artifacts remain by design).
G FlowEdit: 1/11 clean; typical failures: part regrown (stairs, legs, fins, top tier), near-no-op (torso), identity drift (Freddy brown→gold — tex unanchored).
Mechanism: DINO-cond ΔV is too weak to CLEAR voxels (and the skipped early steps are exactly the structural ones); surviving tokens get repainted plausibly by the anchored generation — deletion cannot be left to the model.
2D edits themselves are mostly fine — the loss happens in the 2D→3D lift.
1/11 vs 10/11 — and the causal chain. The fix uses our structural advantage: we KNOW the part voxels; Buffalo has to infer a box.All 11 assets · columns: source | QIE 2D edit | P procedural | G FlowEdit. Rows 3/4/9/11: the removed part visibly regrows in G.
7 · Next: v2 hybrid + Stage-5 filtering
v2 hybrid: procedurally clear part voxels in SS (exact), FlowEdit/anchored gen only heals the cut region; tex anchored from self-encoded source (kills drift). Every failure row above maps onto one of these two fixes.
