Русская версия · Physical x4→x16 mod · Русская инструкция x4→x16 · Benchmarks · Troubleshooting
A beginner-friendly end-to-end guide for NVIDIA CMP 170HX owners.
Recommended path:
stock CMP 170HX → memory/compute unlock → 170tune validation → physical PCIe x4→x16 hardware mod → PCIe Gen2 software unlock → working CUDA P2P → multi-GPU LLM testing
Measured on our 2× CMP 170HX 64 GB system: ~6.46–6.69 GB/s one-way P2P, ~12.90–13.18 GB/s bidirectional, ~1.59–1.65 µs GPU-to-GPU latency over PCIe Gen2 x16.
CMP 170HX modding is experimental. This guide includes patched NVIDIA kernel modules, Linux boot parameters, tuning and an optional hardware soldering modification. Keep physical/remote recovery access and change one thing at a time.
Use the community unlock work first:
The common 8 GB CMP 170HX can expose 64 GB HBM2e after the tested unlock procedure.
Detailed beginner guide: docs/UNLOCK.md
Project:
Use 170tune to qualify HBM/clock changes and detect silent corruption before persistence.
Detailed guide: docs/170TUNE.md
This is a real separate step and must be done before expecting x16 bandwidth.
The CMP 170HX PCB routes all 16 PCIe lanes, but the factory board leaves the coupling capacitors for lanes 4–15 unpopulated. To restore x16, populate 24 missing 0402 AC-coupling capacitors.
Parts successfully used on our cards:
Samsung CL05B224KO5NNNC
0.22 µF / 220 nF
X7R
16 V
0402
Quantity for x16: 24 pieces
ChipDip listing used for our build:
Recommended equipment: microscope/magnification, fine tweezers, good gel flux, fine temperature-controlled iron, solder wick and a multimeter. A controlled bottom preheater can make the job easier, but it is not mandatory; our successful cards were soldered without one. Avoid uncontrolled whole-board heating.
After soldering, verify the real negotiated width:
sudo lspci -vv -s <GPU_BDF> | grep -E 'LnkCap:|LnkSta:'Successful hardware result:
Width x16
With the separate Gen2 software work, our cards report approximately:
LnkSta: Speed 5GT/s, Width x16
Full physical-mod instructions:
- English — complete PCIe x4 → x16 soldering guide
- Русский — полная инструкция по физической переделке x4 → x16
These pages include the parts list, soldering workflow, preheating advice, diagnostics and upstream references.
The capacitor modification changes link width. Gen1 → Gen2 is a separate software change.
Current public research has Gen2 working. Gen3 remains an open research problem.
Our tested P2P path builds on the experimental aikitoria/open-gpu-kernel-modules P2P changes plus our CMP/GA100 mailbox correction:
- https://github.com/aikitoria/open-gpu-kernel-modules/tree/610.43.03-p2p
- Mailbox fix
- P2P installation
- How the P2P fix works
Alternative BAR1-based P2P work is also documented:
Our final CUDA test on Gen2 x16:
GPU0 -> GPU1: 6.46 GB/s
GPU1 -> GPU0: 6.69 GB/s
Bidirectional: 12.90–13.18 GB/s
GPU latency: 1.59–1.65 us
Full results: docs/BENCHMARKS.md
- Install Linux and the supported NVIDIA Open Kernel Module version.
- Verify the card is visible and stable at stock settings.
- Unlock memory/compute with
cmpunlocker. - Cold power-cycle and verify 64 GB / 40 GB geometry.
- Install
170tune, run preflight and validate stability. - Perform the physical PCIe x4 → x16 capacitor modification if x16 is wanted.
- Verify
LnkStareportsWidth x16. - Apply/verify the PCIe Gen2 software unlock and confirm
Speed 5GT/s. - Add the selected P2P path.
- Verify IOMMU/ACS/topology for that P2P method.
- Run
nvidia-smi topo -p2p. - Run
p2pBandwidthLatencyTestand judge P2P by actual bandwidth. - Benchmark the real LLM/CUDA workload.
- Unlock guide
- 170tune guide
- PCIe x4 → x16 physical soldering guide
- Русская инструкция x4 → x16
- PCIe Gen3 research/status
- P2P installation and verification
- P2P alternative paths
- How the P2P failure and fix work
- Benchmarks
- Troubleshooting
- Mailbox patch
- Contribution / reproduction guide
- Raw successful P2P benchmark
- Русская версия
The complete result has currently been validated on one dual-CMP 170HX system. Other driver versions, kernels, motherboards, card revisions and PCIe topologies may behave differently.