feat(inference): add compatible local provider failover

[Pal Lakatos-Toth (pallakatos)]

Publication stack

This is the next bounded public slice after #543 -> #544 -> #545. It targets the standing-team foundation branch so reviewers see only this inference delta. Do not merge into the intermediate publication branch; after the parent lands, retarget to kars-bridge and require fresh qualification. Base repair commits will be propagated forward.

Scope

• Add operator-configured provider endpoints and per-provider credentials, with credential-free local inference.
• Add ordered primary/fallback routes and bounded recovery before request acceptance; do not replay accepted streams.
• Add optional local-inference egress targets and bounded fallback fields without limiting existing team roster sizes.
• Keep existing Content Safety/prompt-shield defaults, public provider guardrails, explicit client model choice, and installation profiles.
• Keep named-provider secrets router-only and preserve existing default routes when no new configuration is supplied.

No mesh task delivery, runtime orchestration, GitHub writes, access-request service, witness deployment, private H100 values or Bridge code is included. Kars remains independently usable without Bridge.

Qualification evidence

Implementation commit df0540c was locally qualified with 2,119 Rust tests, strict Clippy/formatting, 94 CLI compatibility tests (two skipped), Helm lint and four installation profiles. Three existing Rust doctests were ignored. Local CLI used the available Vitest 4.1.10 cache, not locked 4.1.8; exact-lockfile GitHub CI is required on this integrated head. Foundation hash centralization ea6f578 is incorporated in 7a2a5d1, and six static gates passed afterward.

Repair closure

Provider credentials and health caches are isolated by selected identity; named routes never borrow ambient default credentials. Legacy informational primary-provider metadata preserves the default route. Typed forwarding failures distinguish authentication/configuration, pre-acceptance transport failure, known 429/5xx rejection, and accepted responses. Truncated known rejection bodies remain eligible for fallback, but accepted buffered or streaming generations are never replayed. Canonical authorization, inheritance and materialization retain ordered fallback configuration.

The final repair batch passed 84 router, 76 controller and six Helm regressions, plus strict Clippy for both crates. These local results supplement, rather than replace, the current GitHub checks.

Review gates still open

• Automated technical review and bounded regression closure are complete. Genuine independent approval is still required; automated review is not a human sign-off.
• The capability audit document is present at docs/security-audits/2026-09-07-inference-local-failover.md. Its required author and independent-reviewer sign-offs are deliberately pending; no human approvals or second email identity have been fabricated to make CI green.
• The qualified foundation/upgrade repairs are incorporated from their owning PRs; no unrelated fixes are hidden in this inference delta.
• At exact head 45cfa009, all technical CI checks pass; security-audit-required correctly remains blocked on genuine sign-offs. This PR stays draft until the required sign-offs and review gates are satisfied.

No main merge, customer deployment, H100 operation or public image release is performed by preparing this PR.

Integration landing plan

The public integration branch kars-bridge starts at current main 7eb039e9. Land the reviewed, fully qualified slices there in order: #543 -> #544 -> #545 -> #547, followed by the remaining bounded layers. Downstream PRs stay stacked for focused review until their parent lands, then retarget to the integration branch. Preserve merge ancestry; do not squash/restack away reviewed history by default.

The integration branch has required CI/security gates, up-to-date-base checks, independent approval and conversation resolution, with force pushes, deletion and admin bypass disabled. Its CI pushes must not automatically publish images or releases. Required genuine audit sign-offs are not waived.

Only after complete cross-layer review and qualification of standalone Kars, existing-customer CLI/install/upgrade behavior, and additive Bridge install/removal should a final integration-to-main promotion PR be merged. No partial runtime is being landed directly in main.

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[Pal Lakatos-Toth (pallakatos)]

Automated closure update: the independent bounded review reports both remaining inference regressions closed at 0a74306 versus c47418f, with no significant regressions found in that repair delta. The subsequent 45cfa00 merge carries only the parent phase-constant and Helm-test timing corrections.

Legacy metadata-only primary tags retain default routing unless explicit routing is configured. Truncated known 429/5xx rejections can fail over; accepted responses and authentication/configuration failures cannot.

This is automated source review, not human sign-off. Exact-head CI and the capability audit’s genuine author/independent-reviewer signatures remain required; no audit or branch-protection gate is waived.