feat(global-routing): add signed envelope routing and consumer dispatch - #1
Open
skeptrunedev wants to merge 8 commits into
Open
feat(global-routing): add signed envelope routing and consumer dispatch#1skeptrunedev wants to merge 8 commits into
skeptrunedev wants to merge 8 commits into
Conversation
The envelope is the trust boundary between global processing (authentication, canonical normalization, native tokenization, one regional routing decision) and regional execution (pod proxy admission, serving, billing finalization). Detached-JWS wire form: env1.<b64url payload>.<b64url Ed25519 sig>. Verifiers check the signature over the received payload bytes and only then parse, so no cross-language canonicalization exists. The payload carries immutable account identifiers (never a raw API key), the canonical normalized body, the native prompt token digest that the serving frontend must reproduce, the selected region and pool, and a bounded validity window. Claude-Session: https://claude.ai/code/session_01Gq5jeTYZ2SDusjNscnyVF5 Signed-off-by: skeptrune <nick.k@trieve.ai>
Add nvext.prompt_token_digest, stamped by the regional pod proxy from the signed routing envelope, and reject any request whose native preprocessing does not reproduce the routed token sequence. This closes the route-one-prompt-execute-another gap: normalization drift between global processing and a serving frontend now fails closed with a 409 instead of serving silently, and both outcomes are counted in morph_global_routing_execute_exact_total. Claude-Session: https://claude.ai/code/session_01Gq5jeTYZ2SDusjNscnyVF5 Signed-off-by: skeptrune <nick.k@trieve.ai>
Replace shadow observation with an authoritative in-path decision client (DYN_GLOBAL_ROUTER_CONSUMER_URL): the frontend queries the consumer after native preprocessing, fails closed when configured, and mints the signed internal request envelope from the trusted auth metadata that pod-proxy-lifted resolves. The envelope rides request context until dispatch. The decision wire contract moves into dynamo-llm as the canonical types and the consumer re-exports them, adding selected_region so dispatchers can address the target region without a dc-id lookup. Envelope wire form becomes a JSON object with key_id for rotation and a domain-separated Ed25519 signature. Claude-Session: https://claude.ai/code/session_01Gq5jeTYZ2SDusjNscnyVF5 Signed-off-by: skeptrune <nick.k@trieve.ai>
The consumer owns both halves of the routing decision: it evaluates the
policy and it ships the signed internal request. POST /v1/dispatch
verifies the envelope (signature, version, validity window; never
forwarding an envelope it cannot verify), resolves the selected region
against --dispatch-target region=https://url entries that must cover
every configured relay, forwards the identical wire object over
HTTPS/mTLS, and relays the response byte stream back unmodified so
client cancellation propagates hop-by-hop. Addressing is deliberately
left to the target region's own verifier. Dispatch outcomes surface as
global_ckf_consumer_dispatch{es,_rejects,_failures}_total.
Claude-Session: https://claude.ai/code/session_01Gq5jeTYZ2SDusjNscnyVF5
Signed-off-by: skeptrune <nick.k@trieve.ai>
Signed-off-by: skeptrune <nick.k@trieve.ai>
Signed-off-by: skeptrune <nick.k@trieve.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes global CKF decisions authoritative and gives them a signed, verifiable path to the selected region, per the signed-internal-request architecture:
lib/llm/global_routing_envelope): the trust boundary between global processing and regional execution. JSON wire object withkey_idfor rotation and a domain-separated Ed25519 signature over the received payload bytes (no cross-language canonicalization). Carries immutable account identifiers (never a raw API key), the canonical normalized body, the native prompt-token digest, the selected region/pool, and a bounded validity window.nvext.prompt_token_digestis recomputed by the serving frontend after its own preprocessing; any mismatch is rejected with 409 and counted inmorph_global_routing_execute_exact_total- a request can never be routed on one token sequence and execute another.DYN_GLOBAL_ROUTER_CONSUMER_URL, fail-closed when configured) and mints the signed envelope from pod-proxy-lifted's trusted auth metadata. The decision wire contract moves intodynamo-llmas the canonical types; the consumer re-exports them and returnsselected_region.POST /v1/dispatch): the consumer evaluates the policy and ships the envelope - verifies it first (never forwards what it cannot verify), resolvesrouting.selected_regionagainst--dispatch-targetentries that must cover every configured relay, forwards the identical wire object over HTTPS/mTLS, and relays the response byte stream unmodified so client cancellation propagates hop-by-hop.Everything ships dark: no behavior changes unless the new env/flags are set; the pod-proxy-lifted service, pod-proxy envelope verification, and prodproxy simplification land separately in the tab repo.
Open item: an in-flight
global_routing_transportmodule (frontend-side WAN client) is not in this PR; its dispatch target should be the consumer's/v1/dispatchso the consumer stays the single dispatch owner.Validation
cargo test -p dynamo-llm --lib --no-default-features: 2063 passed.cargo test -p global-ckf-consumer: 33 passed, including dispatcher round-trip (verified envelope forwarded verbatim, stream + headers relayed) and rejection paths (foreign signature, expired, unroutable region, unreachable target).cargo clippyclean for the new code (pre-existing warnings untouched).https://claude.ai/code/session_01Gq5jeTYZ2SDusjNscnyVF5