DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generation

Main Contributions

Thin-Shell & Non-Manifold 3D

Leverages multi-chart geometry images to effectively learn thin-shell, non-manifold surfaces — geometry that watertight volumetric models cannot represent.

High Fidelity, End-to-End

Continuous TSDF + differentiable marching squares backpropagate 3D surface losses end-to-end, yielding subpixel-accurate boundaries.

Fast & Lightweight

Diffusion over a compact 32×32 latent generates 3D in ~1.2 s on a consumer GPU — down to CPU-only devices.

DiffGI Pipeline

An input 3D mesh is mapped to a 2D TSDF geometry image and compressed by the DiffGI-VAE into a compact 32×32 latent. The decoder reconstructs the TSDF map, and a Differentiable Marching Squares (DMS) module extracts the 3D surface from it. Because DMS is differentiable, pixel-space losses on the TSDF and position maps — together with a geometry-aware normal rendering loss — propagate end-to-end, from the rendered 3D surface all the way back to the 2D latent. A transformer-based latent diffusion model is then trained on this latent space for conditional generation.

The Effect of Our Representation

DiffGI (ours)

Conventional GI

DiffGI versus a conventional occupancy-based geometry image

The same garment recovered from our DiffGI — a continuous, differentiable TSDF geometry image (left) — versus a conventional geometry image built on a binary occupancy map (right). DiffGI keeps thin shells and boundaries sharp and intact, while the occupancy-based representation leaves them torn and aliased.

GT UV atlas, its 256x256 binary occupancy rasterization with staircase boundary, and the 256x256 TSDF whose subpixel boundary matches the GT

GT chart boundaries in UV space, rasterized at 256×256: binary occupancy snaps to the pixel grid, while our TSDF follows the dashed GT boundary with subpixel accuracy.

Results

Single-View Image-to-3D

From a single front-view image to a complete 3D garment. Against TRELLIS, TRELLIS.2, and GarmageNet, DiffGI recovers cleaner boundaries with far more compact meshes (≈23K vertices on average).

Additional image-to-3D results with DiffGI tensors

Additional image-to-3D results, shown with the corresponding 2D DiffGI tensors.

Image-to-3D on GarmageSet — best geometric accuracy with an order of magnitude fewer vertices.
Method	Avg. Vertices	CD↓	MD (F1)↑	d_H↓	BCD↓
TRELLIS	109K	3.44	0.28	15.38	N/A
TRELLIS.2	380K	11.01	0.27	69.70	12.44
GarmageNet	526K	4.31	0.20	23.76	5.64
Ours (DiffGI)	23K	1.35	0.48	8.42	2.91

Reconstruction Fidelity (DiffGI-VAE)

VAE reconstruction comparison on GarmageSet and ABO

VAE reconstructions on GarmageSet and ABO. DiffGI-VAE produces sharper boundaries and better thin-shell preservation than Omages and GarmageNet.

Reconstruction & encoding fidelity — best scores with the most compact representation.
Method	Rep. Size	ABO CD↓	ABO EMD↓	ABO JSD↓	ABO NC↑	Garmage CD↓	Garmage EMD↓	Garmage JSD↓	Garmage NC↑
Omages	64×64×4	0.89	0.25	0.92	0.89	1.31	0.17	1.79	0.95
GarmageNet	N×72	—	—	—	—	2.19	0.21	32.61	0.94
Ours	32×32×4	0.83	0.23	0.89	0.83	0.46	0.16	1.24	0.96

Label-Conditioned Generation (ABO)

Label-conditioned furniture generation on ABO

DiffGI generates thin frames and open-boundary furniture with fewer staircase artifacts than Omages.

Latent Space Interpolation

Smooth interpolations across furniture categories reveal a well-structured, continuous DiffGI latent manifold.

Peak memory and inference latency — DiffGI scales from servers down to CPU-only edge devices.
Method	Hardware	Peak VRAM (GB)↓	Time (s)↓
TRELLIS-image	RTX A6000 Ada	16.28	4.52
Omages	RTX A6000 Ada	2.49	52.0
Ours-Image	RTX A6000 Ada	3.22	0.80
Ours-Image	RTX 4070 (12GB)	3.22	1.21
Ours-Image	MacBook M4 (CPU)	—	8.52

BibTeX

@inproceedings{shim2026diffgi,
  title     = {DiffGI: Differentiable Geometry Images for High-Fidelity Thin-Shell 3D Generation},
  author    = {Shim, Eungjune and Lee, Hansol and Ju, Eunjung},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026}
}