feat(memory): apply kernel memory tunables before memory-mode runs - #507
Conversation
Merging this PR will not alter performance
|
febfce2 to
a219b76
Compare
a219b76 to
3568cec
Compare
Greptile SummaryThe PR adds best-effort Linux memory stabilization around memory-mode executions and restores reversible host settings after each run. It also centralizes Linux profiling sysctl handling and gives wall-time executors ownership of their restoration guards.
Confidence Score: 4/5The PR is not yet safe to merge because local privileged memory runs still mutate host-global memory settings despite the stated CI-only boundary. MemoryExecutor invokes the tunables unconditionally, and MemoryTunables gates only on passwordless elevation, so a local root or passwordless-sudo run still disables THP and eligible swap and irreversibly drops the local page cache. Files Needing Attention: src/executor/memory/executor.rs, src/executor/memory/tunables.rs
|
| Filename | Overview |
|---|---|
| src/executor/memory/tunables.rs | Implements best-effort THP, swap, and page-cache tuning with drop-based restoration. |
| src/executor/memory/executor.rs | Applies the new tunables guard for the lifetime of each memory-mode execution. |
| src/executor/helpers/linux_sysctl.rs | Moves Linux sysctl handling into a shared helper and adds restoration guards. |
| src/executor/wall_time/executor.rs | Retains changed profiling sysctls on the executor so their original values can be restored. |
| src/executor/tests.rs | Updates wall-time test setup so each executor owns its sysctl restoration guard. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[MemoryExecutor run] --> B[Apply memory tunables]
B --> C[Disable THP]
B --> D[Disable eligible swap]
B --> E[Drop page cache]
C --> F[Run memtrack benchmark]
D --> F
E --> F
F --> G[Drop MemoryTunables guard]
G --> H[Restore swap entries]
G --> I[Restore THP mode]
Reviews (5): Last reviewed commit: "fix(walltime): restore profiling sysctls" | Re-trigger Greptile
77ad6fc to
87a02e9
Compare
art049
left a comment
There was a problem hiding this comment.
I know it was the case before but it hurts readability a bit
dcd2e8f to
758c564
Compare
|
@greptileai review again |
758c564 to
fc2fe71
Compare
fc2fe71 to
5619273
Compare
The sysctl read-then-write-with-sudo primitive is not walltime-specific; memory mode needs it too.
Disables transparent huge pages, sets vm.compaction_proactiveness, vm.swappiness and kernel.numa_balancing to 0, disables swap and drops the page cache, so benchmark repos no longer need a hand-written CI step. Applied only in CI, best-effort: a knob that cannot be set is a warning. swapoff is skipped on zram devices and whenever the swapped pages would not fit in available memory.
Keep the original profiler sysctl values in the walltime executor and restore them when it is dropped. This prevents a local or containerized run from leaving host-global profiling access enabled.
5619273 to
e89df29
Compare
Benchmark repos running
codspeed --mode memorycurrently need a hand-written CI step to stabilise kernel memory behaviour before measuring: disable THP, zerovm.compaction_proactiveness/vm.swappiness/kernel.numa_balancing,swapoff -a, and drop the page cache before each suite.The runner now does this itself at the start of every memory-mode execution.
Behaviour
MemoryTunables::apply()returns a guard holding the previous value of every knob it changed, and restores it on drop — including on the error paths, whereteardown()never runs. Only knobs that were not already at the target are captured, and THP is restored to the exact mode it held (madvisestaysmadvise).swapoff -ais guarded: skipped on zram devices (it permanently resets theirdisksize) and whenever the swapped pages would not fit inMemAvailable. Restoring re-enables the recorded entries one by one, sinceswapon -awould miss a swap file absent from/etc/fstab.run()rather thansetup(), so the page cache is dropped before every suite and--skip-setupdoes not bypass it. The sysctl/THP writes are no-ops on later suites sinceensure_sysctlcompares before writing.Commits
refactor(executor): move linux_sysctl into executor helpers— pure move, the sysctl primitive is no longer walltime-specific.feat(memory): apply kernel memory tunables before memory-mode runsVerification
cargo clippy --all-targetsclean,cargo test --lib(pre-existing failures on this host are the sudo-requiring tests, which writekernel.kptr_restrictand are identical onmain).Not running in CI, skipping kernel memory tunables; host THP unchanged.enabled/defragbecome[never],vm.compaction_proactiveness = 0,vm.swappiness = 0, and the swap guard correctly loggedLeaving swap enabled: swapped pages do not fit in available memory.