Cross-compile to apple-darwin from Linux fails: -MM dep-track uses CPPFLAGS only, no --target/-isysroot

[zackees]

Symptom

Cross-compiling tikv-jemalloc-sys 0.7.1+5.3.1 to x86_64-apple-darwin from a Linux host fails with:

include/jemalloc/internal/jemalloc_internal_decls.h:45:12: fatal error: 'os/lock.h' file not found
   45 | #               include <os/lock.h>
      |                         ^~~~~~~~~~~
1 error generated.
make: *** [Makefile:510: src/jemalloc.sym.o] Error 1

Critically, the compile invocation of clang itself succeeds with -isysroot $APPLE_SDK_PATH -mmacosx-version-min=11.0. The Apple SDK at $APPLE_SDK_PATH/usr/include/os/lock.h is present and resolvable — verified by running the same clang -E src/jemalloc.c command manually outside make. The compile completes cleanly when invoked directly with the exact same CFLAGS.

Root cause

The Makefile jemalloc's configure generates has TWO clang invocations per .c file:

# Makefile:511 (the compile rule)
$(CC) $(CFLAGS) -c $(CPPFLAGS) $(CTARGET) $<
ifdef CC_MM
	@$(CC) -MM $(CPPFLAGS) -MT $@ -o $(@:%.$(O)=%.d) $<
endif

The @$(CC) -MM $(CPPFLAGS) ... (dependency-tracking) invocation:

1. Uses $(CPPFLAGS) only — NOT $(CFLAGS)
2. Is silenced by the @ prefix even with make V=1, making it invisible during debugging

tikv-jemalloc-sys's build.rs invokes ./configure with:

CC=clang
CFLAGS='-O3 ... --target=x86_64-apple-macosx -isysroot /path/to/MacOSX11.3.sdk -mmacosx-version-min=11.0 -fuse-ld=lld'
LDFLAGS=
CPPFLAGS=

CPPFLAGS is set to empty when passing to configure (cc-rs cflags_env() populates only CFLAGS for the per-target flags). Configure then generates a Makefile where CPPFLAGS := -D_REENTRANT -I$(objroot)include -I$(srcroot)include — no sysroot, no target, no SDK include path.

The result: the -MM invocation runs clang -MM -D_REENTRANT -Iinclude src/jemalloc.c with no target hint and no sysroot. clang preprocesses for the host (Linux), looks for <os/lock.h> in /usr/include/os/, fails to find it, aborts.

Reproducer (Linux host targeting darwin)

# Inside a Linux container with clang-18, llvm-18, lld-18, and the
# MacOSX11.3.sdk extracted at /opt/apple-sdk/MacOSX11.3.sdk:

mkdir -p ~/repro && cd ~/repro
cat > Cargo.toml <<'EOF'
[package]
name = "repro"
version = "0.1.0"
edition = "2021"

[dependencies]
tikv-jemallocator = { version = "0.7", features = ["unprefixed_malloc_on_supported_platforms"] }
EOF
mkdir -p src && echo 'fn main() {}' > src/main.rs

rustup target add x86_64-apple-darwin

export CC_x86_64_apple_darwin="clang --target=x86_64-apple-darwin -isysroot /opt/apple-sdk/MacOSX11.3.sdk -mmacosx-version-min=11.0"
export CFLAGS_x86_64_apple_darwin="-isysroot /opt/apple-sdk/MacOSX11.3.sdk -mmacosx-version-min=11.0"
export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=clang

cargo build --target x86_64-apple-darwin
# -> fails at jemalloc_internal_decls.h:45 with `'os/lock.h' file not found`

What workarounds we tried (none sufficient)

1. Setting CPPFLAGS_x86_64_apple_darwin (cc-rs-style per-target env var) — cc-rs doesn't propagate this to configure's CPPFLAGS slot. config.log still shows CPPFLAGS=.

2. Setting plain CPPFLAGS (no triple suffix) before cargo build — tikv-jemalloc-sys's build.rs explicitly sets CPPFLAGS="" when invoking configure, clobbering any env value. config.log still shows CPPFLAGS=.

3. Wrapping clang as a script (/tmp/wrap/clang that exec's the real clang with --target=apple-darwin -isysroot $SDK baked in) and setting CC_x86_64_apple_darwin=/tmp/wrap/clang — cc-rs's compiler detection for cross-targets falls back to bare clang name regardless of the per-target env var. config.log still shows CC=clang.

4. Symlinking /usr/bin/clang AND /usr/bin/clang-18 to the wrapper script — same result. cc-rs's compiler.path() still returns clang (the name, not the resolved path), and configure records CC=clang. The wrapper IS invoked at compile time (because bare clang resolves to it via PATH), but the wrapper's flags get merged into argv such that the -MM invocation's --target is overridden by something downstream.

What would fix this upstream

In tikv-jemalloc-sys/build.rs, when constructing the configure invocation, also pass CPPFLAGS derived from the same per-target sysroot env vars cc-rs already provides. Concretely:

// build.rs around line 160
let cppflags = read_and_watch_env("CPPFLAGS")
    .map(OsString::from)
    .unwrap_or_default();

// NEW: also pull --target + -isysroot from cc-rs's compiler so the
// Makefile's `-MM` invocation has them too.
let target_flags: Vec<&OsStr> = compiler
    .cflags_env()
    .as_bytes()
    .split(|b| *b == b' ')
    .filter(|s| !s.is_empty() && (
        s.starts_with(b"--target=") ||
        s.starts_with(b"-isysroot") ||
        s.starts_with(b"-mmacosx-version-min=") ||
        // ... etc
    ))
    .collect();
// merge into cppflags

OR — simpler — pass CC as the full command line including --target=$T -isysroot $SDK so the bare $(CC) -MM invocation picks it up. cc-rs deliberately doesn't do this, but tikv-jemalloc-sys could.

OR — patch the configure invocation to add -isysroot $SDK --target=$T to the Makefile's CPPFLAGS via --with-jemalloc-cppflags=... or equivalent.

Caller-side workaround

For now, the only reliable workaround we've found is to disable tikv-jemallocator for darwin targets entirely. In the calling Cargo.toml:

[target.'cfg(any(target_os = "linux", target_os = "freebsd"))'.dependencies]
tikv-jemallocator = { version = "0.7" }

…and cfg-gate any #[global_allocator] static GLOBAL: tikv_jemallocator::Jemalloc = ... declarations similarly. macOS uses the system allocator instead. Performance impact is minimal for most workloads.

Context

This is the long-standing blocker for zackees/soldr's macOS x64 release lane. We've gone through ~10 release iterations (v0.7.72 through v0.7.87) trying to make the cross-compile work. The structural fixes we landed (--fuse-ld=lld in CFLAGS, inline Apple SDK fetch, CFLAGS_/CXXFLAGS_ env vars) got past all the OTHER C deps (libz-ng-sys, ring, zstd-sys, bzip2-sys, libmimalloc-sys, libsqlite3-sys) — only tikv-jemalloc-sys remains blocked, because of this -MM/CPPFLAGS gap.

Our docker repro harness at zackees/soldr lives at ci/docker-darwin-cross/ and reproduces the exact failure in 30s-2min iteration loops.

[ti-chi-bot]

Welcome @zackees! It looks like this is your first issue to tikv/jemallocator 🎉