Loader lifecycle edge cases deferred from #289

[terasakisatoshi]

PR #289 (issue #277 Phase B) put every FFI entry point behind one generation snapshot, made a retired image keep its identity, and added scripts/lint_generation_snapshot.sh so a new entry point cannot regress. Five rounds of review closed everything that could corrupt memory or lose an error. What follows is the remainder: real, but each one needs a design decision or a build-shape change rather than another patch, and none of them is a regression introduced by #289.

1. A generic struct's constructor and destructor are two artifacts

Each generic instantiation is monomorphized into its own cdylib, so Boxed<i32>::new and Boxed<i32>_free are normally two different images. The object is therefore allocated by one image and freed through another, which crosses an allocator boundary — the contract docs/src/panics.md sets out for every other path.

#289 made this as safe as it can be without changing the build shape: the constructor resolves the destructor in the same step that allocates, preferring its own image and otherwise capturing the flag of the image the destructor lives in, so the liveness flag always describes the image the finalizer will call into (_call_generic_constructor, generic_struct_generation_snapshot). What it does not do is put them in one image.

The fix: give a generic struct's instantiation one artifact identity — one specialization library per (struct, type arguments), exporting the constructor, the methods and <Struct>_free together — so the allocation and the free are the same allocator, and one flag covers both by construction.

Also missing: reload-stress coverage for generic structs. test/test_hot_reload_transaction.jl hammers plain calls, panics, crate-module reads and non-generic constructors against a reload loop; generic structs are covered only deterministically (test/test_generic_struct.jl), because a generic instantiation compiles a fresh cdylib per type argument and a reload loop over that is minutes of rustc, not seconds. Worth doing behind a slow-test flag once the artifact question above is settled.

2. build.rs inputs cannot be enumerated

_crate_build_cfg_text memoizes the #[cfg] probe on the crate's Cargo.toml, build.rs and .cargo/config.toml chain. A build script can emit a different cargo::rustc-cfg from things no digest sees: an environment variable, a generated file, a sibling crate, the clock.

#289's answer is to make a hot reload re-probe unconditionally (memo = false) and keep the memo for first loads, where the crate has not been built under this process before. That is correct but conservative in one direction and permissive in the other: a first load in a long session can still be answered from a stale memo if the caller rebuilds the crate by other means.

The fix: either take the cfg from the build itself (a --message-format=json pass records the build-script-executed cfgs) rather than a second cargo rustc --print cfg, or drop the memo entirely and measure whether the probe cost matters.

3. Closing is still explicit

Replaced and unloaded images are retired, not closed — a call may be inside one and there is no per-call reader pin. The user reclaims them with unload_library(name; close = true) or close_retired_handles!. So a long-lived process that reloads many times accumulates mapped images until it says otherwise.

The fix, if it is worth it: an epoch or a hazard-pointer scheme that can prove no call is inside a retired image and close it automatically. That is two atomics on every FFI call, which is why #289 did not take it; the trade is worth measuring before choosing.

4. Smaller items

• @rust_llvm keeps its own dlopen and is the single allowlist entry in scripts/lint_load_path.sh and scripts/lint_generation_snapshot.sh. Deprecate @rust_llvm and the LLVM IR integration path (rustc/Julia LLVM version mismatch) #265 Phase 2 removes the path; until then it is outside both rules.
• Generated field accessors and destructors have no panic boundary. They are not emitted by generate_wrapper, so a panic in a user Drop impl still aborts. Documented in docs/src/panics.md.
• Cargo-path specialization of dependency-backed generics stays fail-closed (body_has_cfg): a lazy specialization is a direct rustc build, and reproducing the Cargo build's configuration for it is its own change.

Context: #277, and the review history on #289.

[terasakisatoshi]

From Codex round 8 on #289 (4348f49): same path under two live lib_names returns one handle but mints a second alive flag; unloading either name records only one flag. Apply one-mapped-image-one-flag to the two-live-names case.

[terasakisatoshi]

From Codex round 9 on #289 (6981c2a, src/loadpolicy.jl:1824): when two tasks initialise the same precompiled module concurrently, the second _alias_reloaded_library call finds target already aliasing the same handle and _retire_alive!(target) flips the shared flag to false while the image stays mapped — objects from it then skip their destructor. Make the alias operation idempotent: if the existing target entry has the same handle, return without retiring.

[terasakisatoshi]

PR #298 takes the two contained items from section 4 — _alias_reloaded_library idempotency and the one-flag-per-image invariant when the same path is loaded under two live names — with a test for each, run at 1 and 4 threads on macOS aarch64 and Linux x86_64.

Both turned out to be memory-safety bugs rather than tidiness:

• alias_artifact! retired the target name's flag unconditionally, so re-aliasing a name that already named this image flipped the image's own flag. _alias_reloaded_library runs on every _resolve_lib, so the second call through one precompiled module declared a live library dead: objects inert, destructors never run.
• Loading one path under a second live name minted a second Ref(true) for one image. unload_artifact! retires with the first flag it finds and drops the rest without flipping them, so after close = true every object holding a dropped flag reads true and runs its destructor into unmapped code. load_artifact! now adopts the flag the image already carries (registered_alive_for_handle), which makes "first flag" correct by construction.

Explicitly not in that PR, and still open here:

1. One specialization artifact per generic struct instantiation — a build-shape change, not an edit to src/loadpolicy.jl. Load policy Phase B: one load/registration path, LOCAL visibility, pinned panic, unified finalizers (#277) #289 already made the destructor resolve in the step that allocates, so the flag describes the image the finalizer enters; putting the constructor, the methods and <Struct>_free in one image is the remaining half, along with the reload-stress coverage behind a slow-test flag that depends on it.
2. Taking the #[cfg] set from a --message-format=json build rather than a second cargo rustc --print cfg. Also a build-shape change; the memo-vs-probe-cost measurement it needs has not been done.
3. Automatic closing of retired images — two atomics on every FFI call, and Loader lifecycle edge cases deferred from #289 #291 already says the trade is worth measuring before choosing.
4. The remaining smaller items: @rust_llvm's own dlopen waits on Deprecate @rust_llvm and the LLVM IR integration path (rustc/Julia LLVM version mismatch) #265 Phase 2; the missing panic boundary on generated field accessors and destructors lives in src/structs.jl; Cargo-path specialization of dependency-backed generics stays fail-closed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bqtry6ehnv5YzeEdrTPCTD

[terasakisatoshi]

Phase boundary for #361, per maintainer request: grouped generic images, captured-image members, Cargo-backed specialization (dependency/cfg/config/lockfile replay), and the cfg-probe/reclamation assessment are delivered. Next phase still needs generated destructor/accessor panic boundaries and actual generic hot-reload front-door stress. Registration-replacement tests alone do not satisfy that reload requirement. Keep this issue open after #361.