Skip to content

@rust_crate at a package's top level can be precompiled (#339) - #351

Merged
terasakisatoshi merged 40 commits into
mainfrom
fix/rust-crate-precompile
Sep 8, 2026
Merged

@rust_crate at a package's top level can be precompiled (#339)#351
terasakisatoshi merged 40 commits into
mainfrom
fix/rust-crate-precompile

Conversation

@terasakisatoshi

@terasakisatoshi terasakisatoshi commented Sep 7, 2026

Copy link
Copy Markdown
Member

Summary

A package that writes @rust_crate at its top level could not be precompiled. The macro evaluated the generated module into an anonymous Module under Main, and Julia refuses to serialize a side effect into a module that is not part of what it is compiling:

ERROR: LoadError: Evaluation into the closed module `##RustCallCrateRuntime#277`
breaks incremental compilation because the side effects will not be permanent.

The module is now evaluated inside the module that expands the macro (load_crate_bindings(...; target_module = __module__)), so it belongs to the package's module tree like any other submodule.

What changes for a user

module MyPackage
using RustCall
@rust_crate joinpath(@__DIR__, "..", "deps", "my_crate") submodule="Bindings"
using .Bindings: add, Point
export add, Point
end

Two separate options, deliberately:

  • submodule="X" is new: it defines the generated module in the calling module as Caller.X, which is what makes using .X: ... possible. This is the package idiom, and the same shape as include("generated/Bindings.jl").
  • name="X" is unchanged: it names the generated module and defines nothing. const MyBindings = @rust_crate path name="MyBindings" — the form the macro's docstring has always shown — keeps working, and the constant is the only binding the caller gets.

Without either, the module goes into a hidden per-call child namespace of the caller: nothing the caller did not name appears in its namespace, and repeated calls never collide (the #222 contract). load_crate_bindings called without a target_module keeps the anonymous module — the REPL and a call inside a function are unchanged. The return value is still a CrateBindings, and the package needs RustCall alone among its dependencies (the generated module reaches Libdl through it).

Why the two options are separate. My first cut made name= do both jobs. Codex caught it, and the failure is worse than the "invalid redefinition of constant" it predicted: a package containing const MyBindings = @rust_crate path name="MyBindings" precompiles and then segfaults on load (signal 11), because the constant is bound over the module binding the macro had just created. test/test_rust_crate_precompile.jl now pins that form, in-process and in a precompiled package loaded by a fresh session.

When things happen: the crate is scanned and built while the package is precompiled (nothing is written into the package); the library is opened by the generated module's __init__ at load time, in whatever session loads the package; after RustCall.clear_cache() or a rebuild of the crate the package's cache is stale and the next using re-precompiles it.

Two consequences for the library path, and one bug they uncovered

The module's _LIB_PATH is now the durable library — RustCall's cache copy — instead of the per-process generation copy, which is made in __init__ (as the written-file template has done since format 6): after precompilation __init__ runs in a later session than the one that generated the module. The module declares that library with Base.include_dependency, which is what makes a cleared cache produce a re-precompile rather than a failed dlopen.

Checking that claim turned up a bug that predates this PR: @rust_crate <crate> cache=false on a crate RustCall has to wrap has never worked. Such a crate is built through a wrapper project in a temporary directory that cleanup_cargo_project deletes as soon as the build returns; with caching on the library had already been copied into the cache, with cache=false nothing copied it, and loading failed with

could not load library ".../rustcall_wrapper_XXXXXX/target/release/lib….dylib"

(reproduced on main at 21ab8f2). The library is now taken out of the wrapper project before the cleanup — into the cache, or into a directory of its own — exactly as _build_pyo3_wrapper_project already did for the PyO3 wrapper path.

Tests

test/test_rust_crate_precompile.jl is new. A precompile image is only ever consumed by a session other than the one that produced it, so it drives subprocesses over temporary packages built around test/fixtures/sample_crate, with a cache directory of their own:

acceptance criterion of #339 test
1. the package precompiles, and a second session loads it and calls f(...) / T(...) steps 1 and 2: Base.isprecompiled, add, Point, distance_from_origin, p isa Point, typeof(p) === B.Point
2. the path survives a new session; clear_cache() is deterministic step 2 checks _LIB_PATH is under the cache and that the loaded file is a .rustcall.-stamped sibling copy; the include_dependency record is read out of the cache header; step 3 checks the image goes stale and rebuilds
3. in-memory behaviour is unchanged test_crate_bindings.jl / test_docs_examples.jl keep the #222 contract for name= unchanged, and a new case asserts that without a name only a ##RustCallCrateRuntime#N namespace appears, under the caller and not under Main
4. the naming rule is written down docs/src/crate_bindings.md, "Using @rust_crate inside a package"
5. docs + CHANGELOG + lints crate_bindings.md, precompilation.md, examples/README.md, CHANGELOG; all five scripts/lint_*.sh pass

Plus the cache=false wrapper case, which threw before the fix, and the const X = ... name="X" case from the review.

Closes #339

🤖 Generated with Claude Code

https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-08T15:13:24.909003Z 684543f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 628465244c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5ff06a64f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae115a2996

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b995b96841

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 27f71c52fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@terasakisatoshi

Copy link
Copy Markdown
Member Author

Scope decision

Four Codex rounds on this PR, and the last three findings are one class: the precompile invalidation scheme is file-based, and a crate's artifact identity is not. Each round has been fixed on its own terms —

  1. the library alone did not catch an edit to the crate → the crate's input files are declared;
  2. .cargo/config.toml decides the flags → the effective Cargo configuration is declared;
  3. a file appearing is invisible to file tracking → the directories are declared too, and PYO3_CONFIG_FILE with them;
  4. the environment is not a file at all → the module records it and warns at load time.

Round 4 is where the class stops yielding to this PR: no include_dependency can represent RUSTFLAGS or PYO3_PYTHON, and the obvious workaround — a stamp file whose content is the artifact key — is circular, because only the step being skipped would rewrite it.

So: #355 (milestone v0.3.2) records the class, the failure it produces and the three options a real fix would choose between. This PR merges once CI is green on the head; further findings of that class belong on #355 rather than here.

Every acceptance criterion of #339 has a named test — see the table in the PR body — and the full suite is green locally (9475 tests, 3 known broken).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74a9d1a695

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8c0c7dd09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f4dd9b6134

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/artifact_id.jl Outdated
Comment thread src/crate_bindings.jl
Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70c5fe6a5f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: ccc250e7ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@terasakisatoshi
terasakisatoshi force-pushed the fix/rust-crate-precompile branch from ccc250e to b19703e Compare September 8, 2026 06:39
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b19703eb72

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00c752355f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
Comment thread src/crate_bindings.jl
@terasakisatoshi

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Unknown error
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64c74e4ecd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
… are (#339 review)

When pyo3's own configuration decides the link directory
(`PYO3_CROSS_LIB_DIR`, the `lib_dir` of a `PYO3_CONFIG_FILE`), pyo3
consults no interpreter: `python_link_source()` returns an empty
fingerprint and `_pyo3_wrapper_build_env` keys nothing by what
`PYO3_PYTHON` resolves to. `_recorded_build_env` recorded the resolved
interpreter and its fingerprint anyway, so a `PYTHONHOME` change or a
retargeted shim warned about a stale library — and advised a forced
precompile — that would have selected the same artifact.

The record now takes the plan once (`_python_link_source_or_empty()`)
and uses its fingerprint verbatim, and the resolved interpreter is ""
on the configured branch; `_python_fingerprint` had no caller left.

Test: with a `PYO3_CONFIG_FILE` naming `lib_dir` and `PYO3_PYTHON`
set, the selection is recorded and the resolved/fingerprint records are
empty and equal to the plan's; off that branch the fingerprint is the
plan's. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw
@terasakisatoshi

Copy link
Copy Markdown
Member Author

Scope decision. The last six Codex rounds (b2d09f5f834350) each surfaced one more case of the same class: the load-time environment record a generated module compares in __init__ (_recorded_build_env) over- or under-approximating the artifact identity for one more selector — :python_free plans, python3-config on a framework Python, the effective link directory, and now the interpreter fingerprint on the configured branch. Each was real and each is fixed in this PR with a test, but the class exists because the record is a warning built beside the identity rather than the identity itself, which is exactly what #355 (v0.3.2) is about: an image that is invalidated by non-file inputs needs no record to compare. This round is fixed in f834350; further findings of that class are deferred to #355 and recorded there. Merging on green.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f834350d0d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
…339 review)

With `PYO3_CONFIG_FILE` set, a plain crate with no pyo3 anywhere in its
graph still hashed the file into its key and declared it as a precompile
dependency, so an edit to an unrelated Python configuration rebuilt the
crate under a new registry identity and invalidated every package image
that binds it.

`local_path_dependency_dirs` now returns, memoized with the graph, whether
the build may read the file: `pyo3` / `pyo3-ffi` / `pyo3-build-config` in
the resolved `cargo tree`, or declared — optional or not, a feature may
activate it — in `[dependencies]` / `[build-dependencies]` of any manifest
in the local graph (`[dev-dependencies]` are never built); `true` when
Cargo could not resolve the graph, because a missing input is a stale
library and an extra one a rebuild. `crate_may_read_pyo3_config` gates
both the digest (`_plain_crate_build_env(crate_path)`) and the declared
file.

Tests: the optional-pyo3 fixture declares and keys the file; the plain
sample crate, when Cargo resolved its graph, neither declares it nor
changes its key on an edit. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ff0511821

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/artifact_id.jl Outdated
`py = { workspace = true, optional = true }` says nothing about the
package; the `[workspace.dependencies]` entry it inherits does (`py = {
package = "pyo3", ... }`). `_manifest_declares_pyo3` read only the member's
alias, so with the dependency optional — omitted from the default `cargo
tree` graph — such a member was classified as unable to read
`PYO3_CONFIG_FILE`, and both the key digest and the declared input were
dropped for a build that a feature makes read the file.

The inherited specification is resolved first
(`_workspace_dependency_table`, the same table `_declared_path_dependencies`
reads), then the package name is checked.

Test: two workspaces whose members declare `py = { workspace = true,
optional = true }`, one inheriting `package = "pyo3"` and one
`package = "anyhow"`: only the first declares pyo3 and may read the
configuration (the negative side when Cargo resolved the graph). Full
suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07887070d2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/artifact_id.jl Outdated
…ILE (#339 review)

An optional *registry* dependency that depends on pyo3 is invisible both
to the default `cargo tree` graph, which omits an inactive dependency,
and to the local manifests, which see only its name. A `features = [...]`
build of such a crate reads `PYO3_CONFIG_FILE`, yet
`crate_may_read_pyo3_config` said no, and the key digest and the declared
input were dropped.

The graph `cargo tree --all-features` resolves is a superset of the graph
any feature selection asks for, so it is consulted — one more `cargo
tree`, memoized with the graph, and only for a crate that names pyo3
nowhere — and `true` when it cannot be resolved: what cannot be inspected
is not ruled out.

Test: a crate with an optional `pyo3-ffi` under another name — absent
from the default graph, present in the all-features graph — may read the
configuration; the same crate with an optional `anyhow` may not, when
Cargo resolved the graphs. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b7a314ebb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/artifact_id.jl Outdated
…G_FILE (#339 review)

The graph the directory list comes from is resolved with `normal,build,dev`
edges, and it was also consulted for pyo3 readers, so a crate that keeps
pyo3 only under `[dev-dependencies]` — compiled by `cargo test`, never by
the `cargo build` a binding runs — hashed the file's contents into its key
and declared it, and an edit to test-only Python configuration rebuilt the
library and invalidated the image.

pyo3 readers are now decided by the local manifests (`[dependencies]` /
`[build-dependencies]`) and, when those name none, by the `normal,build`
graph with every feature on (`_build_graph_may_use_pyo3`); the
dev-inclusive listing is no longer consulted for it.

Test: a crate with `pyo3-ffi` under `[dev-dependencies]` shows it in the
dev-inclusive graph and not in the build graph, declares none, and may
not read the configuration. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a06c59556

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
`PYO3_*` is on the #282 allowlist because pyo3's build script reads it —
and only that build script does. For a plain crate with no pyo3 anywhere
in its graph the namespace still went into the key, the registry name and
the load-time record, so configuring Python for another package (a new
`PYO3_PYTHON`, a `PYO3_CONFIG_FILE`) rebuilt this crate under a new
identity and warned about its library for nothing.

`_plain_crate_build_env(crate_path)` drops the namespace when
`crate_may_read_pyo3_config` is false, `_recorded_build_env(; pyo3)` does
the same, and the generated module records the decision
(`_READS_PYO3_CONFIG`) so `__init__` compares the same set it recorded.
Wrapper builds are unchanged: they depend on pyo3 by construction.

Test: the sample crate's key is the same under two `PYO3_PYTHON`s and with
none; neither its build env nor its record carries a `PYO3_*` entry; the
record does with `pyo3 = true`; a changed `PYO3_PYTHON` does not warn for
it. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f3faa1281

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
terasakisatoshi and others added 2 commits September 8, 2026 23:10
…s are gated (#339 review)

Reverts 6f3faa1 and states the line it crossed. Cargo hands every ambient
variable to every build script, and a crate's own `build.rs` may read
`PYO3_PYTHON` without depending on pyo3 — which crates are in the graph
proves nothing about what a script reads. So the allowlist is taken
whole for every plain build, `PYO3_*` included (the #282 contract): a
spare rebuild when Python is configured for another package, never a
stale library.

The *contents* of the file `PYO3_CONFIG_FILE` names stay gated on
`crate_may_read_pyo3_config`: the identity covers declared inputs, and
the one crate that parses that file is `pyo3-build-config` — the same
reason a path dependency's content is hashed because Cargo declares it,
not because a script might `include_str!` it.

Test: the sample crate's key differs under two `PYO3_PYTHON`s, and the
variable is in both its build env and its record. Full suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c7895fff1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl Outdated
terasakisatoshi and others added 6 commits September 8, 2026 23:30
… contents are gated (#339 review)"

This reverts commit c7895ff.
…e variable's value (#339 review)

Reverts the dependency-graph gate (3ff0511, 0788707, 8b7a314, 4a06c59,
c7895ff) and states one rule in its place. Two rounds of review pulled
in opposite directions — gate the digest and the `PYO3_*` values on pyo3
being in the graph, then keep them because a crate's own `build.rs` may
read either without depending on pyo3. The second argument is the one
that describes a stale library rather than a spare rebuild, and it
applies to the file's contents exactly as it applies to the variable's
value: Cargo hands every ambient variable to every build script, and the
graph proves nothing about what a script opens.

So `_plain_crate_build_env()` is the allowlist plus the digest of
`PYO3_CONFIG_FILE`'s contents whenever the variable is set, for every
plain build, and `_crate_precompile_dependencies` declares the file the
same way — as `.cargo/config.toml`'s contents are hashed and declared.
No `cargo tree --all-features`, no reader list, no per-crate predicate.

Test: the sample crate, with no pyo3 anywhere in its graph, keys
`PYO3_PYTHON` and the config file's digest, and declares the file. Full
suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bdb52016dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/crate_bindings.jl
Comment thread src/crate_bindings.jl Outdated
…ig file selected before it exists is seen appearing (#339 review)

Two findings on bdb5201.

**The interpreter, not only `PYO3_*`.** A plain build of a crate that
depends on pyo3 — a `cdylib` with `#[julia]` items — runs pyo3's build
script, which selects an interpreter (`PYO3_PYTHON`, else `python3` on
`PATH`) and configures the library for that Python's ABI. The plain key
held the raw `PYO3_*` values and the config digest, so a `PATH` that now
finds another Python, or a shim retargeted under one name, kept the key
and the load-time record unchanged and a precompiled package loaded a
library configured for the previous ABI. `_pyo3_build_interpreter()` —
the same order pyo3 uses, no interpreter when pyo3's own configuration
decides — is in `_plain_crate_build_env()` under the wrapper identity's
names and in every plain module's record.

**A selected file that does not exist yet.** `PYO3_CONFIG_FILE` naming
an absent path declared nothing, so the file appearing later — with a
build script that tolerated its absence — left the image valid. Until it
exists its directory is the declared input (the entry list sees the
creation), afterwards the file is; and every module's record carries the
contents' digest ("" unset, the unreadable marker when absent), so a
load after the creation is told even when the image survived.

Tests (Unix for the fakes): two fake `python3`s on `PATH` give two keys
and two records, the record warns when `PATH` moves to the other and not
before; `PYO3_PYTHON` is taken as given; a configured `lib_dir` keys and
records no interpreter. An absent selected file tracks its directory,
not itself; creating it flips both and warns. The load-time testsets
that built their "recorded" set from the bare allowlist now build it
from `_recorded_build_env`, which is what a module records. Full suite
green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iJ1dZ7KEebWDjdY7fhPbw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

(lib_name = crate_library_name(info; release = build_release,
features = features, default_features = default_features))

P1 Badge Include the build environment in written-module names

When two packages use write_bindings_to_file for the same plain crate under different RUSTFLAGS, CC, PyO3 configuration, or interpreter fingerprints, they can carry different binaries but receive the same _LIB_NAME because this call omits _plain_crate_build_env(). Loading the second package then replaces the shared registry entry and updates the first generated module's mirror, so the first package can call the other binary with incompatible symbols or ABI; compute one build-environment snapshot and pass it to crate_library_name, as generate_bindings now does. CLAUDE.mdL58-L60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@terasakisatoshi
terasakisatoshi merged commit 11fa2bf into main Sep 8, 2026
29 checks passed
@terasakisatoshi
terasakisatoshi deleted the fix/rust-crate-precompile branch September 8, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@rust_crate at package top level cannot be precompiled: the generated module is evaluated into an anonymous Main module

1 participant