Bpf oom structops - #168
Open
hodgesds wants to merge 2 commits into
Open
Conversation
`memory` has had the seam for this all along: `oom::OomKiller` +
`register_oom_killer`, deliberately policy-free because `memory` cannot
enumerate tasks or deliver signals. `narf-bpf-oom` fills that slot from a
verified BPF program set, in the shape `narf-bpf-idle` established — the
`struct_ops!` macro emits the trait, the adapter, and the cap-gated install
entry; a `#[commit(...)]` committer binds the adapter and hands `memory` a
killer that dispatches through it. `memory` learns nothing about BPF.
Only the *ranking* is programmable. Enumerating tasks, resolving address
spaces, and queuing the SIGKILL need allocation, locks, and pointers no
verified program may hold, so the crate does those natively (`live.rs`) and
asks the program to score candidates over scalars. `init`, kernel tasks, and
tasks with no resident memory never reach a program, so a hostile or broken
policy can change *which* eligible task dies, never whether the kill is legal.
The trait's two exclusions are deliberately unequal, because a program that
traps is indistinguishable from one returning 0 (both yield `DEFAULT_RET`):
* `badness == 0` is soft. If a policy ranks nothing — the shape a buggy or
trapping program takes — selection falls back to native Linux-shaped
badness rather than killing nothing, so no program can switch the OOM
killer off and wedge the machine. Same call Linux's
`bpf_handle_out_of_memory` makes. Counted via `native_fallback_count()`,
never silent.
* `veto` is hard, and survives that fallback: a policy protecting a process
keeps protecting it exactly when the rest of its ranking misfires.
Candidates come from a `CandidateSource` trait rather than a direct call to
`userspace`, so the smokes can rank synthetic tasks and record the kill instead
of delivering it — a suite that ranked live tasks would SIGKILL the shell
running it — and so an out-of-tree crate can supply its own notion of
candidate. The committer refuses to install with no source registered, since
that would arm a killer that can never find a victim.
`CapKind::OomPolicy` (0x020A) is reserved per docs/PLUGGABILITY.md. It is a new
kind rather than a reused one because `memory`'s seam predates the
pluggable-cap convention and is uncapped, and because choosing which process
dies is a kill authority that must not be derivable from `MemAlloc` or `Pager`.
Off by default: `frame`'s `bpf-oom` feature compiles the consumer in and
registers the candidate source; the in-tree `narf_userspace::oom` policy keeps
`memory`'s slot until a program set is installed at runtime.
13 smokes under `bpf/oom` (positive + negative per behaviour): descriptor
shape, all four ctx words in signature order, the program's score deciding the
victim, veto exclusion, the soft/hard fallback split, native ranking with the
`oom_score_adj` opt-out, wrong-cap / missing-method / unknown-method /
sleepable-program rejection each proved not to reach the live slot, the
no-source refusal, and `memory`'s own `request_oom_relief` dispatching through
the installed policy.
Full suite: x86_64 7150 pass / 0 fail / 70 skip; aarch64 5179 pass / 0 fail /
35 skip.
Originating prompt: "lets implement bpf-oom as a struct_ops implementation for
narf. It should be a standalone crate that is able to load various bpf programs
as OOM killer policies. make a separate branch for this work and make sure
there are appropriate tests" (with follow-ups: only if it makes sense to do;
make it a kernel crate feature; fall back when the BPF program exits).
Model: Claude Opus 5 (1M context).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`badness` took four scalars — pid, RSS, `oom_score_adj`, total pages — and that was not a design choice, it was `MAX_CTX_WORDS`. A struct_ops method packs its arguments into the context tuple, the tuple is four words, and `badness` used all four. Everything else a policy might reasonably weigh (how much of the victim is actually resident, how much is writable anonymous memory reclaim cannot get back, eventually which cgroup it is in) was unreachable by construction. A **region ctx** removes the ceiling by reusing the packet mechanism for something that is not a packet: the program enters with `(data, data_end)` over a read-only structure the hook fills in, and reads fields after proving each read in bounds. `PtrKind::Mem`/`MemEnd` is already the verifier's general model of a dynamically-bounded region, so this adds no verifier rule — only a second `TypeKey`, distinct from the packet key so a region pointer is not interchangeable with a packet one and the XDP adjust intrinsics cannot reach it. Unlike `XDP_CTX`, `data` is READONLY: a hook context describes state the program is being asked about, so a store through it fails verification. narf-bpf: `load_region_ctx` verifies against that shape, `run_atomic_region` binds the region per invocation. Each refuses the other kind and `run_atomic`/`run_atomic_interpreted` refuse a region-ctx program, so no path leaves a bounded pointer unbacked. Atomic-only, for the same reason a typed probe object is — nothing keeps the borrow alive across an await. narf-bpf-structops: a trait declares `#[ctx_region]` and every method takes one `&C where C: CtxStruct`, an unsafe trait whose contract is `#[repr(C)]`, padding-free, pointer-free plain data — the structure is copied verbatim into a buffer a hostile program reads, so padding would be a stack disclosure and a pointer an address leak. The adapter passes that *copy*, so even a verifier bug admitting a store could only scribble on a stack temporary. `StructOpsDesc::region_ctx` records the shape and `validate` rejects a program loaded for the other one (`WrongCtxShape`): without it a mismatched binding installs cleanly and then answers every call with `DEFAULT_RET`, the same silent failure the sleepable check exists to prevent. A second macro arm rather than an optional attribute because the two forms generate different method bodies and `macro_rules!` has no `else`. narf-bpf-oom: `OomCtx` — pid, tid, rss_pages, oom_score_adj, total_pages, mapped_bytes, resident_pages, writable_nonexec_bytes. Eight fields where four words fit. The live source now takes one `memory_stats()` block per candidate instead of calling `mapped_bytes()` (which is that block's first field), so the extra fields cost nothing. Fields are append-only, asserted by a const layout check: a loaded program keeps reading the offset it was compiled against, and a program compiled against a wider struct than the kernel publishes fails its own bounds check and takes its fallback path rather than reading past the end. Two new smokes, and the existing ones now read fields through the bounds-check prologue rather than reading ctx words: every one of the eight offsets is proved to arrive where `ctx::offset` says (the failure the scalar form could not even express), a read one word past the end is proved bounded, an unproved dereference and a dereference of `data_end` are proved to fail *load*, and a scalar-ctx program bound to this trait is proved rejected at install. The sleepable smoke now asserts both gates — `load_region_ctx` refuses it outright, and install still rejects it on context if it was loaded another way. Full suite: x86_64 7152 pass / 0 fail / 70 skip; aarch64 5181 pass / 0 fail / 35 skip; 15 `bpf/oom` smokes on both. bpf spec §3.16 documents the mechanism. Originating prompt: "do the ctx struct" (following a question about whether BPF iterators were needed — they are not; the kernel owns the candidate walk, so what was missing was context width, not iteration). Model: Claude Opus 5 (1M context). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.