Clean SWH20 · Complete Image-to-3D Evaluation

TRELLIS.2-4B vs Hunyuan3D-2.1 vs S3-T50 · 2026-08-18

image-to-3DevaluationSWHgeometrytexture

TL;DR

S3-T50 and TRELLIS.2 form the top geometry tier in this pilot; S3-T50 leads 9/10 aggregate texture columns after coordinate and remesh bugs are fixed.

A raw-OBJ coordinate regression invalidated the original HY3D table; the repaired geometry-stage score raises S3 Uni3D-I from 0.285 to 0.362.

This is a clean 20-case pilot, not a publication-scale claim. Scale to at least 200 frozen clean cases; use SWH50 only as an intermediate probe.

1 · Frozen suite

20 deterministic Software Heritage assets; TexVerse is excluded.
The same held-out input image is passed to every model.
Every track has a complete 20 × 3 matrix and a strict validator.
Decoded shape/PBR latents are versioned proxies; they are not raw source meshes.

The suite separates semantic E2E alignment, paired geometry fidelity, and fixed-shape texture fidelity.

2 · End-to-end alignment

Model	N	CLIP ↑	CLIP-N ↑	ULIP color ↑	Uni3D color ↑	HY3D ULIP ↑	HY3D Uni3D ↑
TRELLIS.2-4B	20	0.8145	0.7817	0.4535	0.3982	0.2041	0.3686
Hunyuan3D-2.1	20	0.8201	0.7833	0.4915	0.4113	0.2005	0.3605
S3-T50	20	0.8197	0.7817	0.4666	0.3917	0.1998	0.3623

Hunyuan leads CLIP-N and the colored full-asset ULIP/Uni3D metrics.
TRELLIS.2 leads the coordinate-canonicalized HY3D geometry-stage metrics.
The repaired HY3D geometry-stage means place all three models in a narrow range.
S3 and TRELLIS.2 means are nearly identical on CLIP-N.

Final asset	N	HY3D ULIP ↑	HY3D Uni3D ↑
TRELLIS.2-4B	20	0.1890	0.3505
Hunyuan3D-2.1	20	0.2002	0.3607
S3-T50	20	0.1864	0.3420

The separate delivered-asset audit keeps export/remesh effects visible; Hunyuan leads that gray-point table.

Broken-axis bars expose the small E2E gaps: Hunyuan leads the colored full-asset metrics, while TRELLIS.2 leads both HY3D geometry-stage means.

The same-name shape.obj changed from presentation coordinates to decoder-native Y-up; fixed adapters recover S3 and TRELLIS.2 Uni3D scores.

Canonical render/point sanity grid used to audit orientation, occupancy, and color before scoring.

3 · Geometry

Model	N	Chamfer-L1 ↓	F@0.05 ↑	F@0.1 ↑	Normal ↑	HD95 ↓
TRELLIS.2-4B	20	0.0416	0.9254	0.9703	0.8000	0.0878
Hunyuan3D-2.1	20	0.0583	0.8262	0.9343	0.7230	0.0997
S3-T50	20	0.0455	0.9070	0.9642	0.8119	0.0895

TRELLIS.2 has the best mean Chamfer, F-scores, and HD95.
S3 has the best normal-consistency mean.
Bold table values and black-outlined bars mark the best mean in each metric.

Broken y-axes omit the shared baseline: bold labels and black outlines mark the top mean—TRELLIS.2 leads four metrics and S3 leads normal consistency.

4 · Geometry fairness and scope

Fairness check	Status	Evidence	Remaining limit
Same condition	PASS	Identical image per asset and model	Single deterministic view
Same scored artifact	PASS	Geometry-stage shape mesh for all models	Final-export effects are a separate audit
Same scorer	PASS	Shared 100k GT samples, deterministic equal-area sampling	Nearest-neighbor surface metrics
Alignment	PASS	Frozen centroid/scale + rigid ICP; no per-case rotation search	Pose fidelity is intentionally removed
GT independence	CAUTION	One shared decoded SWH target	TRELLIS.2-family decoder proxy may favor TRELLIS/S3 representations
Statistical power	PILOT	Paired analysis completed	N=20 is not publication scale
Train overlap	PARTIAL	S3 exact overlap is 0/60 against audited v4 manifests	External model manifests are incomplete

Protocol-internal comparison is fair for intrinsic complete-shape fidelity. It is not evidence that S3 is globally SOTA.

Geometry question	Current SWH surface track	Pixal3D single-view track	Coverage
Complete 3D surface	Chamfer, F-score, Normal, HD95	Not measured directly	Implemented
Visible-view silhouette	Removed by 3D alignment	Normal-render IoU	Not implemented
Visible normal fidelity	3D nearest-neighbor \|cos θ\|	Mean/median/boundary angular error	Not implemented
Pixel-level discrepancy	Not measured	PSNR, SSIM, LPIPS on normal maps	Not implemented
Pose / projection fidelity	Mostly removed by ICP	Preserved in the input camera frame	Not implemented
Hidden/back-side geometry	Measured	Mostly unobserved	Implemented

The two tracks answer complementary questions and must remain separate scorecards.
Current numbers cannot be compared directly with Pixal3D Table 1.
The next paper-aligned addition is Toys4K GT-camera normal rendering.

5 · Texture on fixed shape identity

Model	N	CLIP-I ↑	LPIPS ↓	PSNR ↑	SSIM ↑	Dataset FID ↓	Dataset CLIP-FID ↓	Dataset CMMD ↓
TRELLIS.2-4B	20	0.9588	0.1447	15.5554	0.5867	122.7496	9.1840	0.0811
Hunyuan3D-2.1	20	0.9504	0.1779	13.8961	0.5245	132.2301	10.7787	0.0689
S3-T50	20	0.9623	0.1186	16.8077	0.5904	106.1915	7.9348	0.0707

S3 leads 9/10 aggregate texture columns; Hunyuan leads dataset-level CMMD.
Bold values mark the best model in each metric column.
All model-to-canonical minimum silhouette IoUs exceed 0.982; the frozen gate is 0.90.

After protocol repair, S3 leads all reference-similarity columns and two of three dataset distribution metrics.

Canonical atlas examples expose color placement directly; gray pixels are invalid/unobserved atlas regions.

Strict GT/model-to-canonical silhouette gates catch coordinate and remesh confounds before any texture score is accepted.

6 · Runtime and audit status

Model	N	Median sec/asset	Mean sec/asset
TRELLIS.2-4B	20	24.74	27.35
Hunyuan3D-2.1	20	67.72	72.85
S3-T50	20	56.41	63.13

TRELLIS.2 uses its official 512 pipeline and is the fastest model in this run.
S3 exact overlap with audited v4 image/text training manifests: 0/60.
TRELLIS.2/Hunyuan unseen status is externally asserted because complete public train manifests are unavailable.
All generation, E2E, geometry, texture, and silhouette validators pass.

Median model inference only; suite rendering and metric encoders are excluded from per-model runtime.

7 · Next gates

Scale the frozen paired protocol to at least 200 clean test cases; use SWH50 only as an intermediate cross-source probe.
Add the Pixal3D Toys4K GT-camera normal-render track without replacing the complete-surface track.
Add multiple seeds only if stochastic variance is part of the claim.
Keep raw-source and decoded-latent GT scorecards separate.
Do not report a single weighted overall score without a user-validated weighting study.

WilD3DGen evaluation suite · clean SWH20 · seed 0 · all validators passed