rtx-p2p

Unlock PCIe peer-to-peer on GeForce RTX 5090 with a single undocumented GSP firmware registry key — stock NVIDIA driver, zero kernel patches.

---

What

Two modprobe options that enable PCIe P2P DMA between consumer GeForce cards, replacing the community approach of patching open-gpu-kernel-modules (aikitoria / geohot fork):

options nvidia NVreg_RegistryDwords="RMDisableFeatureDisablement=1;EnableResizableBar=1"

• No kernel module rebuild
• Works with the stock NVIDIA driver
• One modprobe drop-in + a driver reload

Why it works

RMDisableFeatureDisablement is a registry key that exists only inside the GSP firmware binary — not in any public NVIDIA source tree, not in nvrm_registry.h, not in any driver fork. When set, it bypasses the GSP-side GeForce SKU feature gate and a cluster of firmware crash guards (ebreak traps) that would otherwise halt the GSP firmware when a consumer GPU touches professional-tier code paths. P2P setup is one of those paths.

EnableResizableBar=1 resizes BAR1 from 256 MB to the full VRAM — BAR1 is the P2P data aperture.

Together, those two flags are sufficient. Full reverse‑engineering write-up: docs/RMDISABLEFEATUREDISABLEMENT.md.

Quick start

sudo ./scripts/enable-p2p.sh
./scripts/verify-p2p.sh

enable-p2p.sh writes modprobe.d/nvidia-p2p.conf into /etc/modprobe.d/, kills anything holding /dev/nvidia*, and reloads the kernel module. verify-p2p.sh prints BAR1 size, nvidia-smi topo -p2p, and the cudaDeviceCanAccessPeer matrix.

To revert:

sudo ./scripts/disable-p2p.sh

Prerequisites

• NVIDIA proprietary or open-kernel driver, 590.x or later (tested on 590.48.01)
• Consumer Blackwell GPU (tested on RTX 5090 / GB202)
• Re-Size BAR enabled in motherboard UEFI / BIOS
• On AMD systems, IOMMU in passthrough mode: add amd_iommu=on iommu=pt to the GRUB kernel command line
• Multiple GPUs on the same host for P2P to be meaningful

Repository layout

modprobe.d/     NVIDIA modprobe drop-in (the two registry keys)
scripts/        enable / disable / verify helpers, cupy peer-bandwidth probe
docs/           GSP firmware reverse-engineering notes

How this was found

Ghidra + Capstone RISC-V reverse engineering of the GSP firmware (gsp_elf_9.bin, extracted from driver 590.48.01). The GSP runs a RISC-V 64 core inside the GPU and parses NVreg_RegistryDwords via a giant registry parser function (FUN_0101d664, ~12 KB of code, ~850 keys). Cross-referencing the keys with feature-gate checks and ebreak trap sites identified RMDisableFeatureDisablement as the single flag that bypasses the SKU gate and the init-time crash guards.

See docs/RMDISABLEFEATUREDISABLEMENT.md, docs/IS_GEFORCE.md, and docs/REGISTRY_KEYS.md.

Related keys (not required, but relevant)

Key	Effect
RMForceP2PType=1	Force PCIe P2P path (skip NVLink/C2C detection)
RMPcieP2PType=1	Force BAR1 mailbox P2P mode
ForceP2P=0x11	Force-set the P2P read/write enable bits
NvLinkPeerThroughL2=1	Route peer traffic through local L2 (undocumented, untested)
RmEnableLocalizedMemory=1	NUMA-aware peer memory (undocumented, untested)
RmCeExposeAllLce=1	Expose all copy engines to user space

The first three are redundant once RMDisableFeatureDisablement=1 + EnableResizableBar=1 are set; they just skip detection paths that already succeed. The last three are speculative and documented in docs/REGISTRY_KEYS.md for reference.

License

MIT. See LICENSE.

Disclaimer

Educational reverse-engineering research. No NVIDIA source, firmware, or binary is redistributed here — only observations about its behaviour. Use on hardware you own. Flipping undocumented firmware flags can destabilise the GPU; there is no warranty of any kind.