Skip to content

@rust_crate at package top level cannot be precompiled: the generated module is evaluated into an anonymous Main module #339

Description

@terasakisatoshi

Symptom

A package that binds a crate with @rust_crate at module top level — the shape docs/src/precompilation.md describes as "using @rust_crate directly" — cannot be precompiled:

module MyPkg
using RustCall
const Bindings = @rust_crate joinpath(@__DIR__, "..", "deps", "my_crate")
end
julia --project=. -e 'using Pkg; Pkg.precompile()'
...
ERROR: LoadError: Evaluation into the closed module `##RustCallCrateRuntime#277` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `##RustCallCrateRuntime#277` with `eval` during precompilation - don't do this.
Stacktrace:
  [1] eval(m::Module, e::Any)
    @ Core ./boot.jl:489
  [2] _instantiate_runtime_bindings
    @ ~/RustCall.jl/src/crate_bindings.jl:2091 [inlined]
  [3] load_crate_bindings(crate_path::String; output_module_name::Nothing, build_release::Bool, cache_enabled::Bool, features::Vector{String}, default_features::Bool)
    @ RustCall ~/RustCall.jl/src/crate_bindings.jl:2127
  [4] top-level scope
    @ ~/RustCall.jl/src/crate_bindings.jl:2190

(RustCall main at 8dd9ba4, Julia 1.12.7, macOS; reproduced with examples/SampleCratePyO3Only.jl's crate — the crate kind does not matter.)

Cause

macro rust_crate expands to load_crate_bindings(path; ...), which calls _instantiate_runtime_bindings:

function _instantiate_runtime_bindings(bindings_expr::Expr)
    runtime_namespace = Module(gensym(:RustCallCrateRuntime))
    return Base.invokelatest(Core.eval, runtime_namespace, bindings_expr)
end

The generated module is evaluated into a fresh anonymous Module whose parent is Main. That module is not part of the package being precompiled, so Julia refuses the side effect. The macro has __module__ — the module that expands it — and never uses it.

Two facts make this a bug rather than a limitation:

  • write_bindings_to_file's module is precompile-safe by construction: the file is included by the package, so the module is a child of the package module. The in-memory macro emits the same module (emit_crate_module) and only differs in where it is evaluated.
  • rust"""...""" emits its definitions into the calling module (__module__) and precompiles fine.

A second, related defect: the emitted module stores _LIB_PATH = loadable_library_copy(...), the process-private generation copy (<lib>.rustcall.<host>.<pid>.<n>.<ext>, swept when the process is gone). Even if the module were serialized, its __init__ would look for a file that does not exist in the next session. The written-file template already does the copy inside __init__ (format 6, #309); the in-memory one must do the same.

Acceptance criteria

  1. A package whose src/ contains, at top level, @rust_crate <crate path> name="Bindings" followed by using .Bindings: f, T precompiles (Pkg.precompile() exits 0) and, in a second Julia process, using it loads the library and both f(...) and T(...) work. Tested with a temporary package over test/fixtures/sample_crate.
  2. The library path the precompiled module carries survives a new session (it is RustCall's cache path, copied per process in __init__), and after RustCall.clear_cache() the behaviour is deterministic and documented: the package is re-precompiled and the crate rebuilt on the next using (the library is declared with Base.include_dependency), never a dlopen failure on a path that is gone.
  3. The existing in-memory behaviour is unchanged: @rust_crate at the REPL and inside a function still returns a CrateBindings value, the proxy semantics (test_crate_bindings.jl, test_docs_examples.jl) hold, and without name= nothing visible is defined in the calling module (the [codex] redesign rust crate explicit bindings #222 contract).
  4. The module naming rule is written down: where the generated module lives, what name= does, and what a second @rust_crate of the same crate in the same module does.
  5. docs/src/crate_bindings.md gains a "Using @rust_crate inside a package" subsection with the precompile story; CHANGELOG ### Fixed entry; lints and the docs build pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:crate-bindings@rust_crate, write_bindings_to_file, generated Julia wrappersbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions