feat(kubernetes): add sidecar supervisor topology

[TaylorMutch]

Summary

Adds the Kubernetes sidecar supervisor topology after the combined-topology base from #2074.
The default combined topology remains unchanged, and sidecar is opt-in.

sidecar moves pod-level network enforcement and gateway forwarding into a dedicated supervisor sidecar. The agent container can run as the resolved sandbox UID/GID with runAsNonRoot, no privilege escalation, and all Linux capabilities dropped.

This draft intentionally includes the prerequisite UID/GID commits from #1973 for now. We expect to rebase those out after that work lands separately.

Runtime validation status:

• sidecar has been validated on GKE Autopilot clusters created with gcloud container clusters create-auto --workload-policies=allow-net-admin; the Google Cloud CLI reference lists allow-net-admin as the supported Autopilot workload policy: https://docs.cloud.google.com/sdk/gcloud/reference/container/clusters/create-auto#--workload-policies
• sidecar is experimental with Kata Containers.
• sidecar is known to fail with gVisor because it depends on pod-local network rule setup.

Sidecar mode preserves gateway session and SSH behavior, but intentionally runs the process supervisor in network-only mode. Filesystem policy, process privilege dropping, and process/binary identity checks are not applied in this mode.

Related Issue

References #1827, #981, #899, #1305.

Related PRs: #1973, #2074, #2016.

Changes

• Include prerequisite sandbox UID/GID support: numeric sandbox identities, configurable Kubernetes sandbox UID/GID values, and OpenShift SCC annotation fallback.
• Add the sidecar supervisor topology and the related processEnforcement and proxyUid configuration.
• Render sidecar-mode sandbox pods with a network init container, non-root supervisor sidecar, and unprivileged agent container.
• Add process-supervisor network-only behavior for sidecar mode while keeping SSH/session relay behavior intact.
• Add sidecar e2e Helm values and Skaffold profile support.
• Add a sidecar topology row to the branch Kubernetes E2E matrix.
• Document topology choice, architecture diagrams, permission model, RuntimeClass validation status, and network-only tradeoffs.

Testing

• git diff --check origin/main..feat/kubernetes-sidecar-topology-v2
• cargo check -p openshell-driver-kubernetes -p openshell-sandbox -p openshell-supervisor-process -p openshell-supervisor-network
• cargo test -p openshell-driver-kubernetes --lib
• cargo test -p openshell-supervisor-process --lib
• mise run helm:test
• markdownlint-cli2 docs/kubernetes/topology.mdx docs/kubernetes/setup.mdx
• ruby -e 'require "yaml"; YAML.load_file(".github/workflows/branch-e2e.yml"); puts "ok"'
• mise run rust:lint
• Prior validation from feat(kubernetes): add sidecar and proxy-pod topology configurations #2016: HELM_K3S_LB_HOST_PORT=18080 mise run e2e:kubernetes:sidecar
• Prior validation from feat(kubernetes): add sidecar and proxy-pod topology configurations #2016: sidecar topology smoke-tested as experimental on Kata Containers; known to fail on gVisor.
• Sidecar topology validated on GKE Autopilot using --workload-policies=allow-net-admin.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2076.docs.buildwithfern.com/openshell

[github-actions]

Label test:e2e applied, but pull-request/2076 is at {"messa while the PR head is 6276de3. A maintainer needs to comment /ok to test 6276de38ac92b65b0b24a68b8d09a5eed06f50d7 to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

[mrunalp]

Some comments with help of agent

1. Central trust boundary — the workload can use the sandbox's gateway identity .
Client mTLS and the SA token render 0440 with fsGroup=sandbox_gid, the process leaf skips supervisor-identity isolation in network-only, and the 127.0.0.1:18080 forward is always open. Since the workload child also runs with sandbox_gid, it can read those credentials and reach the gateway directly.

2. nftables fence only covers TCP/UDP.
The output chain is policy accept and only rejects TCP/UDP from non-proxy_uid UIDs. ICMP (e.g. unprivileged ping sockets) and other L4 protocols fall through. CAP_NET_RAW is dropped so raw sockets aren't available, but this is a narrower egress boundary than combined mode's per-netns setup. Low severity — suggest a brief comment in generate_sidecar_bypass_ruleset noting the intentional scope.

3. processEnforcement=full does not restore network binary-identity.
supervisor_sidecar_env hard-codes NETWORK_BINARY_IDENTITY=relaxed regardless of process_enforcement, so full restores agent-container Landlock/privilege-drop but network policy stays endpoint/L7-scoped. The docs are technically accurate ("combined-mode process/filesystem controls"), but a reader could reasonably infer full ≈ combined. A one-line note that network policy remains endpoint-scoped in sidecar mode regardless of processEnforcement would prevent that.

4. Doc drift in the debug skill.
debug-openshell-cluster/SKILL.md says the sidecar runs "with primary GID 0" to read the token, but the implementation uses runAsGroup: sandbox_gid + fsGroup: sandbox_gid (which re-owns the projected volume to sandbox_gid). Please update to match.

[TaylorMutch]

1. Central trust boundary — the workload can use the sandbox's gateway identity .
   Client mTLS and the SA token render 0440 with fsGroup=sandbox_gid, the process leaf skips supervisor-identity isolation in network-only, and the 127.0.0.1:18080 forward is always open. Since the workload child also runs with sandbox_gid, it can read those credentials and reach the gateway directly.

Addressed this by moving the supervisor and sandbox to run with different GIDs.

Updated the other areas with clarifying comments.

[mrunalp]

Deployed this branch on an OpenShift 4.22 cluster (K8s 1.35) with supervisor.topology=sidecar and TLS + mTLS enabled (default PKI init job). The sidecar pod renders correctly, but openshell-network-init is stuck in CrashLoopBackOff:

Error:   × failed to create sidecar directory /etc/openshell-tls/proxy/client
  ╰─▶ Permission denied (os error 13)

$ oc -n openshell get pod test-sidecar
NAME           READY   STATUS                  RESTARTS
test-sidecar   0/2     Init:CrashLoopBackOff   6

Root cause

In run_network_init (crates/openshell-sandbox/src/main.rs) the ordering is:

1. prepare_sidecar_directory(sidecar_tls_dir /* /etc/openshell-tls/proxy */, proxy_uid, proxy_gid, 0o755) — this chowns the TLS dir to proxy_uid (e.g. 1337), mode 0755.
2. copy_sidecar_client_tls_if_present(...) → prepare_sidecar_directory(dest_dir /* …/proxy/client */, …) → create_dir_all.

Step 2 runs as root with capabilities.drop: ["ALL"] (the init container only adds NET_ADMIN, NET_RAW, CHOWN, FOWNER, so it has no CAP_DAC_OVERRIDE). The parent dir is now owned by uid 1337, mode 0755, so root is neither the owner nor has group-write → EACCES. Root can no longer bypass DAC checks because DAC_OVERRIDE was dropped.

Note the state dir was already changed to 0o2775 (setgid + group-write), but the TLS dir is still 0o755, and it's the one that gets a subdirectory created in it right after being chowned away from root.

[TaylorMutch]

@mrunalp I reproduced locally and pushed a fix. Please have a look when you get a chance!