Fix out-of-bounds UDF read causing non-deterministic mesh corruption

[rwfsmith]

Summary

get_vertex_val() in src/remesh/simple_dual_contour.cu uses the result of a hashmap lookup as an array index without checking whether the lookup succeeded. On a miss this reads far out of bounds of udf.

This is undefined behavior on all platforms — it isn't Windows- or Blackwell-specific. It just happens to be visible when the out-of-bounds memory contains something that flips a comparison.

The bug

uint32_t idx = linear_probing_lookup(hashmap_keys, hashmap_vals, key, N_vert);
return udf[idx];

linear_probing_lookup signals "not found" with a sentinel (src/hash/hash.cuh):

if (prev == std::numeric_limits<K>::max()) {
    return std::numeric_limits<V>::max();
}

With V = uint32_t that's UINT32_MAX, so a miss makes the line above evaluate udf[4294967295] — a read roughly 16 GB past the base pointer.

A miss is reachable in normal use: get_vertex_val is called for all 8 corners of each voxel, and corners just outside the voxelized narrow band were never inserted into the hashmap.

Worth noting this is the only call site that dereferences the result directly. The two call sites in src/hash/hash.cu just store the returned value and let the caller handle the sentinel, which suggests this one is an oversight rather than an intentional invariant.

Impact

Whatever garbage lands in idx's slot participates in the isosurface crossing test, so it can inject spurious dual-contouring vertices and edge intersections. Downstream in TRELLIS.2 this showed up as to_glb(..., remesh=True) non-deterministically producing shredded/spiky meshes — the same code and the same latents would give a clean mesh on one run and a corrupted one on the next, because the result depended on whatever happened to be in that memory at the time.

That non-determinism is what makes it worth fixing rather than working around: it's silent, intermittent, and very easy to misattribute to the model.

Fix

Treat a missing corner as "far outside the surface" so the edge simply produces no intersection, instead of reading out-of-bounds memory:

if (idx == std::numeric_limits<uint32_t>::max()) {
    return 1e6f;
}

9 added lines (mostly the explanatory comment), one file, no API or behavior change for corners that are present.

Verification

Reproduced and confirmed fixed on Windows 11 / CUDA 13.0 / RTX 5090 via TRELLIS.2's to_glb(remesh=True): corrupted exports across repeated runs before, consistently clean meshes after. Extracted GLBs check out (e.g. 808k verts / 973k faces, 0 NaNs, 6 degenerate faces, sane extents).

I have not built on Linux, but the change is platform-independent and only adds a bounds guard on a path that was previously UB.