Skip to content

Callbacks: pass Julia functions to Rust as extern "C" fn #296

Description

@terasakisatoshi

Split out of #246, whose other three acceptance criteria are about string marshalling. Callbacks are a feature, not a bug in existing marshalling, and they need a design decision that the string work does not.

Summary

There is no way to pass a Julia function to Rust. juliatype_to_rust (src/typetranslation.jl) has no Function or function-pointer case, src/ffi_contract.jl has no ABI kind for one, and there is no @cfunction marshalling anywhere in the package. So an idiomatic FFI shape — a comparator, a predicate, a progress callback, a visitor — cannot cross the boundary at all:

#[julia]
pub fn apply(f: extern "C" fn(i64) -> i64, x: i64) -> i64 { f(x) }
apply(x -> x * 2, 21)   # no way to express this today

Design sketch

The contract row. A new ABI kind — one C slot, Ptr{Cvoid}, ownership :owned_by_julia — for extern "C" fn(A...) -> R in argument position only. The extractor already parses the Rust signature; the manifest needs to carry the parameter and return types of the function pointer so the Julia side can build the right @cfunction without re-parsing Rust (issue #264's rule). Return position stays unsupported: a Rust function returning a function pointer into Julia has no owner.

Julia side. @cfunction needs the signature at macro-expansion time, so the wrapper generator — which has the manifest — emits @cfunction($f, R, (A...)) from the recorded types. A closure needs the Base.@cfunction/CFunction form (@cfunction $f ...), which returns a heap-allocated CFunction object rather than a bare pointer.

Lifetime. Start with the synchronous-borrow case only: Rust may call the pointer while the FFI call is on the stack, and not after. The CFunction is then rooted for the duration by the same GC.@preserve the string arguments already use (_string_arg_plan), and nothing outlives the call. Storing the callback in Rust — a registered handler, a Box<dyn Fn> kept in a struct — is the case that needs an explicit user-managed handle, and should be a documented second step, not silently allowed by the same syntax.

Thread safety. A @cfunction may only be entered on a thread the Julia runtime knows about. If Rust calls the pointer from a thread it spawned itself, the process dies. So: document that the callback must be invoked on the calling thread, and reject (or at least document as unsupported) any Rust signature that could hand the pointer to another thread — which the type system cannot tell us, so this is a documentation-and-example boundary, not a check.

Panic boundary. This is the interesting interaction. Today the generated extern "C" wrapper runs the Rust body inside catch_unwind and reports a panic through a thread-local channel (#244). A callback inverts the direction: a Julia exception thrown inside the @cfunction would unwind through Rust frames, which is undefined behaviour — Rust's extern "C" boundary is not an unwind boundary for a foreign runtime. The generated @cfunction therefore has to catch every Julia exception itself, stash it, and return a sentinel; the outer wrapper re-raises it after the FFI call returns, in the same shape guard_rust_panic_ptr already uses for the other direction. The two channels must not be confused: a Rust panic inside a callback-driven call and a Julia exception from the callback are different failures with different owners.

Identity and caching. A @cfunction pointer is not part of an artifact's identity (src/artifact_id.jl) — it is a runtime value — but the signature is, because it changes the generated wrapper.

Acceptance criteria

  • A Rust function taking extern "C" fn(i64) -> i64 can be driven by a Julia closure, with a named test
  • A Julia exception thrown inside the callback surfaces as that exception at the call site, and never unwinds through Rust
  • The CFunction is provably rooted for the duration of the call (a test that allocates hard between the creation and the call)
  • Documented: the synchronous-borrow restriction, the thread restriction, and what to do instead when Rust must store the callback

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:codegenrustcall_core, proc-macro, extractor, manifestenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions