[nanvix] E: Embed C/C++ runtime in python.elf for .so dlopen support

[esaurez]

Summary

Updates Makefile.nanvix so python.elf correctly serves as the "main module" against which extension .sos (numpy, ssl, lxml, future pip-installed wheels) resolve their C and C++ runtime symbols at dlopen() time. Without this change, even pure-Python plus pre-compiled extension wheels fail because the static linker drops every libc/libstdc++/libgcc symbol that cpython itself doesn't reference — symbols those .sos definitely need at runtime.

This is the consumer-side companion to two upstream Nanvix changes:

• nanvix/nanvix#2450 — dlfcn loader STB_WEAK support. It makes the loader handle unresolved weak undefined symbols per the System V ABI (return value 0, do not error). This PR ensures python.elf actually presents the strong symbols .sos need so the weak-undef fallback is only used for the genuinely-optional cases.
• nanvix/nanvix#2451 — libposix pathconf / fpathconf ENOSYS stubs. Without these, the cpython ./configure conftest fails ("C compiler cannot create executables") because libstdc++'s std::filesystem::current_path leaves pathconf as an unresolved strong UND.

What changed

File	Change
Makefile.nanvix	Three coordinated CONFIGURE_ENV link-flag changes (details below) + a ~30-line comment block documenting the rationale.
.nanvix/config.py	The unused configure_env() helper is updated to mirror the new Makefile.nanvix flags, with a docstring marking it as currently-unused. (A separate small PR can delete the helper entirely.)

LIBS segment 1 — --whole-archive block

Before:

LIBS="-Wl,--start-group $(LIBPOSIX) $(LIBC) $(LIBM) -lsqlite3 -lssl -lcrypto -lz -lbz2 -llzma -lffi -Wl,--end-group"

After:

LIBS="-Wl,--whole-archive $(LIBPOSIX) $(LIBC) $(LIBM) $(LIBSTDCXX) $(LIBGCC) -Wl,--no-whole-archive -Wl,--start-group -lsqlite3 -lssl -lcrypto -lz -lbz2 -llzma -lffi -Wl,--end-group"

The first segment forces every object from libposix, libc, libm, libstdc++, and libgcc into python.elf. Combined with the already-present -Wl,--export-dynamic flag, this means all of those runtime symbols end up in python.elf's .dynsym and are visible to extension .sos at dlopen() time. Without --whole-archive, the static linker drops unreferenced objects (e.g. fscanf, longjmp, strtold_l for numpy; operator new/delete[], __cxa_*, _Unwind_*, std::type_info vtables for any C++ extension) and subsequent dlopen() fails with "symbol not found".

LIBS segment 2 — trimmed --start-group

The trailing --start-group ... --end-group is now just the external add-on libraries (sqlite3, ssl, crypto, z, bz2, lzma, ffi). It no longer re-lists libposix / libc / libm — those archives are already fully embedded by segment 1, so the external libs can resolve their references against the already-included objects.

New top-level Makefile vars

LIBSTDCXX := -lstdc++
LIBGCC    := -lgcc

Defined once at the top level (the -l form works identically in both the docker and host build paths). The GCC driver resolves them against its built-in search paths; this avoids hardcoding the versioned lib/gcc/i686-nanvix/<gcc-version>/libgcc.a path, which would be fragile across toolchain upgrades.

LDFLAGS — kept identical

LDFLAGS="-L$(SYSROOT_PATH)/lib -T$(SYSROOT_PATH)/lib/user.ld -Wl,--allow-multiple-definition -no-pie -Wl,--export-dynamic -Wl,--no-dynamic-linker"

No change to LDFLAGS. The -Wl,--allow-multiple-definition flag is the only piece that looks workaround-shaped, and the surrounding comment block has been expanded to honestly enumerate the duplicate-symbol categories it masks:

• Newlib long-double math helpers — frexpl, llrintl, lrintl, rintl are defined in three different newlib directories simultaneously (a newlib build-system bug we will file as a discussion issue at nanvix/newlib).
• libposix vs. libc — _start, copysign[f], getenv, setenv, unsetenv, environ, isatty.
• libc vs. libm — frexp, ldexp, modf, isnan, isinf, scalbn, ...
• libm vs. libstdc++ — hypotf.
• libgcc internal — __x86.get_pc_thunk.* duplicates.
• libc vs. libgcc — __eprintf.

The set is large and toolchain-build-version-dependent; treating the link as multiple-definition-tolerant is the only practical workaround until each upstream is fixed. The comment explicitly marks the flag as temporary and lists the categories so a future reader can audit which contributing upstreams have been addressed.

Validation

• CPython 3.12 builds cleanly with the new flags on local-nanvix/toolchain-python:from-prs (a toolchain image built from the filed-PR branches of nanvix/newlib and nanvix/gcc, with no manual workarounds).
• python.elf runs hello.py on the Nanvix microvm without regression.
• numpy 1.26.4 end-to-end: import numpy, np.arange(10).sum(), np.dot(matrix, vector), reshape, flatten, broadcasting all work; the test harness prints NUMPY_TEST_OK.
• Single-process / multi-process / standalone modes: linker changes are not mode-conditional, so the existing test matrix is preserved. (E2E validation done in standalone microvm; the other two modes share the same python.elf.)
• make -f Makefile.nanvix clean still works (cleanup logic doesn't reference the new variables).

Compatibility

• No public API change.
• No source code change to CPython itself (only to the Nanvix build harness).
• python.elf grows by ~3-5 MB because of the additional whole-archive objects (libstdc++ in particular is sizeable). This is the trade-off for .so extensions to work without per-extension RPATHs or static-link sharing.
• Removing the change reverts to a python.elf that runs pure Python correctly but cannot dlopen() any extension .so that depends on libc/libstdc++/libgcc symbols cpython doesn't itself reference (i.e. essentially every real extension wheel).

Future cleanup (not in this PR)

1. nanvix/newlib libm duplicates — file the discussion issue and land the build-system fix, then drop -Wl,--allow-multiple-definition here.
2. nanvix/cpython .nanvix/config.py — delete the unused configure_env() helper entirely. Tracked locally as a follow-up small PR.
3. nanvix/toolchain-python — drop -Wl,--allow-shlib-undefined from the .so link line now that the loader handles weak undefs at runtime (planned follow-up PR against nanvix/toolchain-python).

[Copilot]

Pull request overview

This PR updates the Nanvix CPython build harness so python.elf exports the full C/C++ runtime surface needed for extension modules (.so) to resolve symbols against the main executable at dlopen() time, preventing load-time “symbol not found” failures caused by the static linker discarding unreferenced runtime objects.

Changes:

• Update Makefile.nanvix link flags to --whole-archive libposix/libc/libm plus -lstdc++/-lgcc, while keeping add-on libs in a separate --start-group.
• Add detailed in-file documentation explaining why --export-dynamic, --whole-archive, and --allow-multiple-definition are used.
• Mirror the new link flags in .nanvix/config.py’s (currently unused) configure_env() helper and document its status.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
Makefile.nanvix	Embeds C/C++ runtime archives into python.elf via --whole-archive and documents linker-flag rationale to support .so dlopen() symbol resolution.
.nanvix/config.py	Keeps the unused configure_env() helper’s link flags aligned with Makefile.nanvix and clarifies that Makefile.nanvix is authoritative.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot's findings

• Files reviewed: 2/2 changed files
• Comments generated: 0 new

[ppenna]

Closing as superseded by #782 and the rewritten #690/#732 stack. The current SDK owns libc/compiler runtime composition; the old manual Newlib/libgcc --whole-archive model is no longer a valid link path. The remaining shared-extension work is covered by those PRs and #799, so no additional issue is needed.