Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
c8c1164
feat(security): OIDC identity token verification for trusted publishing
dawsontoth Aug 14, 2026
97e0902
feat(security): hdb_oidc_trust table and trust policy operations
dawsontoth Aug 14, 2026
63e026c
feat(security): exchange_oidc_token — mint a token from a CI identity
dawsontoth Aug 14, 2026
1f27a6c
feat(cli): exchange a CI identity for a Harper token automatically
dawsontoth Aug 14, 2026
08c7ceb
refactor(security): cut comments the code can carry itself
dawsontoth Aug 14, 2026
568cfbf
fix(security): keep identity and refresh tokens out of the operations…
dawsontoth Aug 14, 2026
b0bfa96
refactor(security): make the OIDC core issuer-agnostic, GitHub a profile
dawsontoth Aug 14, 2026
7ddbf0f
test: remove the JWT keys the exchange suite writes, and share the he…
dawsontoth Aug 14, 2026
2e2dd0c
feat(security): narrow a minted token to a subset of its user's opera…
dawsontoth Aug 14, 2026
ff04730
fix(security): carry an empty operation scope instead of dropping it
dawsontoth Aug 14, 2026
4cf3881
fix(security): apply the token operation scope to the SQL path too
dawsontoth Aug 14, 2026
b66736f
fix(security): carry the token operation scope across credential minting
dawsontoth Aug 17, 2026
1e84eaf
fix(security): deny a scoped token from minting a login token
dawsontoth Aug 17, 2026
9daa298
refactor(security): extract the token-scope carry-forward into one he…
dawsontoth Aug 17, 2026
86799c2
fix(security): gate the token scope on the API operation, not the han…
dawsontoth Aug 17, 2026
c8d429b
fix(security): gate the token scope on the job op for non-SQL export …
dawsontoth Aug 17, 2026
a0d4e62
fix(cli): drop an unused path import left by the rebase conflict reso…
dawsontoth Aug 17, 2026
d8740b7
fix(security): harden token scoping per deep-review (4 findings)
dawsontoth Aug 18, 2026
41042ec
fix(security): close credential-minting, scope, and lifetime gaps fro…
dawsontoth Aug 18, 2026
2afa5a6
fix(security): key replay on the signed input, and fix issuer/audienc…
dawsontoth Aug 18, 2026
562fa75
fix(security): bootstrap the replay table and carry the job's real op…
dawsontoth Aug 18, 2026
0286fb7
fix(security): reject a non-boolean `enabled`, and pin exact claim ma…
dawsontoth Aug 18, 2026
db6a5a9
fix(security): act on the SQL permission denial processAST was discar…
dawsontoth Aug 18, 2026
6487950
refactor: split the processAST guard fix out to its own PR (#2202)
dawsontoth Aug 18, 2026
602cdc6
fix(security): never read the SQL scope's operation from the request …
dawsontoth Aug 18, 2026
42315f8
fix(upgrade): patch is_hash_attribute on the replay table; pin the en…
dawsontoth Aug 18, 2026
ce1f999
docs(upgrade): record the TTL-on-first-use limitation for the replay …
dawsontoth Aug 18, 2026
14bcc6f
test: assert the export-job scope gate enforces, not just computes
dawsontoth Aug 18, 2026
ced27a5
test: make the #2202 ordering constraint fail loudly instead of livin…
dawsontoth Aug 18, 2026
10256f4
test: decouple the #2202 tripwire from the status literal it watches
dawsontoth Aug 18, 2026
9c322ed
fix(security): refuse malformed stored policies; let exchange_oidc_to…
dawsontoth Aug 18, 2026
2ebb8e0
fix(security): back off after a failed JWKS refresh; stop overclaimin…
dawsontoth Aug 18, 2026
d8d826c
fix(security): apply the JWKS backoff to an issuer with no cached keys
dawsontoth Aug 18, 2026
be34107
fix(security): tell the truth in list_oidc_trust; pin the guards the …
dawsontoth Aug 18, 2026
9436f50
fix(security): report a deleted or deactivated policy user in list_oi…
dawsontoth Aug 18, 2026
d5a1821
test: pin the invalid_reason precedence a row with both problems reli…
dawsontoth Aug 18, 2026
5da87ee
test: pin the stale-key grace ceiling with an injected clock
dawsontoth Aug 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,33 @@ System tables replicate by default. To opt out, add the name to `NON_REPLICATING

If the table needs `audit: true`, set it both in the schema (for fresh installs) **and** on the `CreateTableObject` instance in the directive (for upgrades) — otherwise the two paths diverge.

## OIDC trusted publishing (`security/authn/oidc/`)

`exchange_oidc_token` lets a workload authenticate with no stored Harper credential (#2171): it presents an identity token minted by its runtime, and gets back a one-hour operation token for the user a stored trust policy names. It is in `NO_AUTH_OPERATIONS` because it _is_ the authentication, the same way `create_authentication_tokens` is against a password — the same three wiring points apply (`serverHandlers.js` `NO_AUTH_OPERATIONS`, the `verifyPerms` bypass in `serverUtilities.ts`, and a `permission(false, [])` registration).

**The core is issuer-agnostic; everything issuer-specific lives in `providers/`.** That split is the point of the layout, not an accident of it — a new workload-identity issuer should be a profile, not a change to verification, matching, or storage.

- `claims.ts` — matching and constraint _shape_ validation. Knows nothing about any issuer.
- `jwks.ts` — issuer keys. The rate-limit clock for unknown-`kid` refetches lives _outside_ the cache entry: a successful fetch replaces the entry, and a rate limit that resets whenever it fires is not a rate limit. Keeping it separate also means a genuine key rotation is picked up on first use rather than after the window.
- `identityToken.ts` — signature, issuer, audience, `exp`, and a bounded lifetime. Owns `rejectToken`, shared with the exchange so both halves refuse identically.
- `tokenExchange.ts` — policy selection, replay, minting, audit. Verification is memoized per audience, so N policies sharing one cost one signature check.
- `providers/` — `assertPolicyIsSpecific` / `assertAudienceIsSpecific` / `normalizeClaims` / `describePrincipal` / optional `vetoClaims`, resolved by normalized issuer.

**An unregistered issuer gets `providers/generic.ts`, which is strict rather than permissive:** the policy must pin `sub`. That is what makes Kubernetes service accounts, GCP service accounts, and SPIFFE SVIDs work with zero provider code — each has a stable canonical subject. GitHub needs its own profile precisely because its `sub` is the one claim you should _not_ pin: it varies by trigger, and its format changed for repositories created after 2026-07-15.

Four constraints that look like choices but are not:

1. **Every rejection returns the same message.** The endpoint is unauthenticated; a caller told which check failed can enumerate a policy one claim at a time. Reasons go to the `oidc-trust` logger.
2. **A GitHub policy must gate the ref.** `githubActionsProfile.assertPolicyIsSpecific` rejects a policy pinning only repository + workflow, because anyone who can push a branch could then add that workflow to it and mint a token. Stricter than npm's trusted-publishing model, which mitigates the same hole with environment protection instead — and profile-scoped, so it never constrains another issuer.
3. **`createOperationToken`, not `createTokens`.** `createTokens` overwrites `hdb_user.refresh_token` as a side effect, so minting for CI would silently revoke whatever credential that user already held (#2018) — the exact problem this feature removes.
4. **The role is the boundary; the per-policy `operations` allowlist only narrows it.** Least privilege is primarily the role of the user the policy names. A policy may _optionally_ carry an `operations` scope, which can only subtract from that role — never add to it. It is deliberately not merged into `permission.operations`: gate 2 in `operation_authorization.ts` treats an explicit listing of an SU-only operation as a deliberate grant, so reusing that field would _widen_ where this must only narrow. The scope is carried as a separate `tokenOperations` claim and intersected ahead of every early return, including the super_user bypass.

Its enforcement surface is the operations API and SQL (`verifyPerms` / `verifyPermsAST`) — **not** the application REST/GraphQL resource path, which authorizes through table-level `checkPermission` and does not consult the scope. A scoped token therefore still carries its role's full CRUD there, which is why the role has to be least-privilege on its own; the scope is defense in depth, not a substitute. Closing that gap is a follow-up on the same surface as CORE-3061. Because a second authorization mechanism beside roles is one more place for the two to disagree, whether to keep this at all is an open design question on #2173 rather than a settled constraint.

Naming `sql` in a scope grants the SQL interface, not unrestricted DML through it: a write statement additionally requires its matching data operation (`insert`/`update`/`delete`) in scope. That is what keeps `read_only` — which expands to include `sql` — from admitting a DELETE, given that `verifyPermsAST` returns early for a super_user before any table check runs.

`hdb_oidc_token_use` (created lazily via `table()`, not the system schema) records spent tokens keyed on a SHA-256 of the token itself, with `expiresAt` past the token's own expiry. Hashed rather than stored, so the table never holds a credential; keyed on the token's **signed input** (`header.payload`) rather than `jti` because not every issuer emits one (Azure uses `uti`). Not on the whole token string: the signature segment is covered by nothing, and base64url decoding ignores the surplus low bits of its final character, so 16 distinct spellings of an RS256 signature decode to the same bytes, all verify, and all hash differently — one leaked token would buy 16 exchanges. ES\* malleability (`s → n−s`) is a second such vector. The signed input is exactly what the issuer asserted, so every variant collapses to one fingerprint. The get-then-put is not atomic and does not claim to be: a concurrent replay is not a privilege escalation, since whoever holds the token could obtain one operation token anyway.

## Table drops, the `dropping` tombstone, and ghost tables

A table is a set of RocksDB column families (`T/` plus `T/<attr>`) and a set of catalog rows
Expand Down
55 changes: 46 additions & 9 deletions bin/cliOperations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as envMgr from '../utility/environment/environmentManager.ts';
envMgr.initSync();
import * as terms from '../utility/hdbTerms.ts';
import { httpRequest } from '../utility/common_utils.ts';
import { workloadIdentityAvailable, exchangeWorkloadIdentityForToken } from './workloadIdentity.ts';
import * as fs from 'fs-extra';
import * as YAML from 'yaml';
import { Readable } from 'node:stream';
Expand Down Expand Up @@ -55,11 +56,11 @@ const TRANSPORT_ONLY_FIELDS = new Set([
'by_ref',
'ref',
'credential',
// `deploy setup=true`'s token, read off the parsed request by deploySetup and sealed locally. No
// operation takes a *top-level* `token`, so keeping it out of every body costs nothing and means a
// mistyped `setup` — which parses as a bare word and falls through to a real deploy — can't carry a
// PAT to the server. Distinct from `credentials[].token`, which is nested inside a field that IS
// sent (ingestCredentials seals it server-side) and is only kept out of the operations log.
// `deploy setup=true`'s token, read off the parsed request by deploySetup and sealed locally.
// Stripping it means a mistyped `setup` — which parses as a bare word and falls through to a real
// deploy — can't carry a PAT to the server. Distinct from `credentials[].token`, which is nested
// inside a field that IS sent (ingestCredentials seals it server-side) and is only kept out of the
// operations log. See OPERATIONS_TAKING_A_TOKEN for the one operation this must not apply to.
'token',
]);

Expand Down Expand Up @@ -179,10 +180,24 @@ async function* wrapPackagingStream(stream: Readable, projectPath: string): Asyn
// Build the JSON operation-field set from `req`, dropping the CLI's internal (`_`-prefixed)
// and transport-only fields so neither the CLI internals nor credentials leak into the
// request body. Shared by the multipart and legacy-JSON deploy body builders.
/**
* Operations whose own request body has a top-level `token`, which must therefore survive the
* transport-only stripping above.
*
* `exchange_oidc_token` (#2171) is one: the identity token IS the request. The blanket strip was
* written when no operation took a top-level `token`, and left this one reaching the server without
* the field it requires — so the issuer-agnostic path was unusable through the generic CLI even
* though direct HTTP worked. Keyed on the operation rather than dropping the strip, because the
* mistyped-`setup` case it guards against is real.
*/
const OPERATIONS_TAKING_A_TOKEN = new Set(['exchange_oidc_token']);

function operationFields(req: any): any {
const keepsToken = OPERATIONS_TAKING_A_TOKEN.has(req?.operation);
const fields: any = {};
for (const [key, value] of Object.entries(req)) {
if (key.startsWith('_') || TRANSPORT_ONLY_FIELDS.has(key)) continue;
if (key.startsWith('_')) continue;
if (TRANSPORT_ONLY_FIELDS.has(key) && !(key === 'token' && keepsToken)) continue;
fields[key] = value;
}
return fields;
Expand Down Expand Up @@ -764,7 +779,8 @@ export async function resolveRequestOptions(req: any): Promise<{ options: any; t
options.timeout = SSE_OPERATIONS.has(req.operation) ? SSE_OPERATION_TIMEOUT_MS : CLI_OPERATION_TIMEOUT_MS;
// Authentication precedence: explicitly configured credentials (dedicated args, URL
// userinfo, env vars) beat everything, then env-var tokens, then the saved `harper login`
// token, and only then the legacy `username=`/`password=` payload fallback below. The
// token, then a CI identity token exchanged via OIDC (#2171 — ambient, so it ranks below
// everything configured), and only then the legacy `username=`/`password=` fallback. The
// tokens must outrank that fallback: for add_user/alter_user those args are the credentials
// of the user being created/altered, so treating them as auth would authenticate as a user
// who doesn't exist yet (or as the wrong identity) instead of using the admin's session.
Expand Down Expand Up @@ -799,9 +815,12 @@ export async function resolveRequestOptions(req: any): Promise<{ options: any; t
const envRefreshToken = tokenPrefix ? process.env[`${tokenPrefix}_REFRESH_TOKEN`]?.trim() : undefined;
// A namespace that is set but blank is a broken CI secret, not a request to fall back to
// whatever the developer last logged in as — say so rather than switching identity silently.
if (tokenPrefix && !envOperationToken && !envRefreshToken) {
const tokenNamespaceBlank = !!tokenPrefix && !envOperationToken && !envRefreshToken;
if (tokenNamespaceBlank) {
console.error(
`Ignoring empty ${tokenPrefix}_OPERATION_TOKEN/${tokenPrefix}_REFRESH_TOKEN; falling back to saved login credentials.`
`Ignoring empty ${tokenPrefix}_OPERATION_TOKEN/${tokenPrefix}_REFRESH_TOKEN; falling back to saved ` +
`login credentials. Workload identity is deliberately NOT used here: a blank token namespace is a ` +
`failed secret, and deploying as a different identity would hide that.`
);
}

Expand All @@ -819,6 +838,24 @@ export async function resolveRequestOptions(req: any): Promise<{ options: any; t
if (tokens.operation_token) {
options.headers.Authorization = `Bearer ${tokens.operation_token}`;
}
} else if (workloadIdentityAvailable() && !tokenNamespaceBlank) {
// Last credential source: no configured token, but this runner can prove its identity to
// the cluster directly (#2171). Deliberately below the env-var and saved tokens — an
// explicitly configured credential should keep working exactly as it did when someone adds
// `id-token: write` to a workflow, rather than silently switching which identity deploys.
//
// `!tokenNamespaceBlank` extends that invariant to the half-configured case. A CI secret
// that failed to populate leaves the namespace set but empty; without this, such a run
// would quietly deploy as the OIDC policy's user instead of failing, which is the same
// silent identity switch in a shape that is harder to notice.
// Standard operation timeout, not the caller's — by this point `options.timeout` may carry
// the 10-minute SSE timeout for a streaming deploy_component, and the exchange is a small
// fast request. Without the override a stalled exchange hangs the deploy for ten minutes,
// on the very operation this feature exists to serve. Same reasoning, same fix as
// refreshExpiredOperationToken above.
const exchangeOptions = { ...options, timeout: CLI_OPERATION_TIMEOUT_MS };
const operationToken = await exchangeWorkloadIdentityForToken(exchangeOptions, target.resolvedTarget);
Comment thread
dawsontoth marked this conversation as resolved.
if (operationToken) options.headers.Authorization = `Bearer ${operationToken}`;
}
}
// Legacy fallback for operations where `username=`/`password=` genuinely ARE the caller's
Expand Down
119 changes: 119 additions & 0 deletions bin/workloadIdentity.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/**
* Client half of OIDC trusted publishing (#2171): ask the runtime for a workload identity token
* addressed to this instance, trade it for a short-lived operation token via `exchange_oidc_token`.
*
* Structured as a provider list because the runtimes differ only in how the token is obtained.
* GitHub Actions is entry one; a Kubernetes entry is `available()` testing for a projected
* service-account token path and `requestToken()` reading that file. A runtime none of them
* recognizes falls through to the CLI's other credential sources.
*/

import { httpRequest } from '../utility/common_utils.ts';

interface WorkloadIdentityProvider {
name: string;
/** True when this process can obtain a token from this runtime. */
available(): boolean;
/**
* Obtains an identity token bound to `audience`. Binding it is what makes the token unusable
* anywhere else — see SHARED_DEFAULT_AUDIENCE in security/authn/oidc/providers/githubActions.ts
* for what an unbound one costs.
*/
requestToken(audience: string): Promise<string>;
}

/** GitHub sets both of these on a job that declares `permissions: id-token: write`. */
const GITHUB_TOKEN_REQUEST_URL = 'ACTIONS_ID_TOKEN_REQUEST_URL';
const GITHUB_TOKEN_REQUEST_TOKEN = 'ACTIONS_ID_TOKEN_REQUEST_TOKEN';

const IDENTITY_REQUEST_TIMEOUT_MS = 10_000;

const githubActions: WorkloadIdentityProvider = {
name: 'GitHub Actions',

/**
* Both variables are required: GitHub sets them together, so their absence means the workflow did
* not grant `id-token: write` — a configuration answer, not a failure to report here.
*/
available(): boolean {
return Boolean(process.env[GITHUB_TOKEN_REQUEST_URL] && process.env[GITHUB_TOKEN_REQUEST_TOKEN]);
},

async requestToken(audience: string): Promise<string> {
const requestUrl = new URL(process.env[GITHUB_TOKEN_REQUEST_URL] as string);
requestUrl.searchParams.set('audience', audience);

const response = await fetch(requestUrl, {
headers: {
authorization: `Bearer ${process.env[GITHUB_TOKEN_REQUEST_TOKEN]}`,
accept: 'application/json',
},
signal: AbortSignal.timeout(IDENTITY_REQUEST_TIMEOUT_MS),
});
if (!response.ok) throw new Error(`GitHub returned ${response.status} for an identity token`);

const body: any = await response.json();
if (typeof body?.value !== 'string' || body.value === '') {
throw new Error('GitHub returned no identity token value');
}
return body.value;
},
};

const PROVIDERS: WorkloadIdentityProvider[] = [githubActions];

function activeProvider(): WorkloadIdentityProvider | undefined {
return PROVIDERS.find((provider) => provider.available());
}

/** True when this runtime can prove its own identity to the cluster. */
export function workloadIdentityAvailable(): boolean {
return activeProvider() !== undefined;
}

/**
* Trades a workload identity token for a Harper operation token, or undefined when this runtime has
* no identity to offer. Failures are reported and swallowed: this is the last credential source
* before the request goes out unauthenticated, and the resulting 401 says nothing useful, so the
* reason is worth printing even though it is not by itself fatal.
*/
export async function exchangeWorkloadIdentityForToken(options: any, audience: string): Promise<string | undefined> {
const provider = activeProvider();
if (!provider) return undefined;

console.error(`Requesting a ${provider.name} identity token for ${audience}...`);
let identityToken: string;
try {
identityToken = await provider.requestToken(audience);
} catch (error) {
console.error(`Could not obtain a ${provider.name} identity token: ${(error as Error).message}`);
return undefined;
}

try {
const response = await httpRequest(options, { operation: 'exchange_oidc_token', token: identityToken });
Comment thread
dawsontoth marked this conversation as resolved.
if (response.statusCode === 200) {
const data = JSON.parse(response.body);
if (data.operation_token) {
console.error(`Authenticated as '${data.username}' via OIDC trust policy '${data.policy}'.`);
return data.operation_token;
}
console.error('The OIDC exchange returned no operation token.');
return undefined;
}
if (response.statusCode === 401) {
// The server deliberately does not say which check failed, so point at the two things the
// operator can actually inspect rather than inventing a cause.
console.error(
'Harper rejected the identity token. Check that a trust policy matches this workload ' +
'(list_oidc_trust) and that its audience is this instance; the server log records the reason.'
);
return undefined;
}
console.error(`OIDC exchange failed: ${response.statusCode}`);
return undefined;
} catch (error) {
console.error(`Error exchanging the identity token: ${(error as Error).message}`);
return undefined;
}
}
9 changes: 9 additions & 0 deletions components/mcp/tools/operations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,11 @@ export const DEFAULT_ALLOW: readonly string[] = [
* `mcp.operations.allow` (an explicit allow list replaces the default and is
* not filtered by this set), where the audit-log redaction of
* `value`/`values`/`envelope` applies.
*
* OIDC trust policies (#2171) are excluded on the same grounds and then some: a
* policy grants an external CI workflow the right to authenticate as a Harper
* user, so reading the set tells an attacker exactly which repository and
* workflow to compromise, and writing one is a way to grant itself access.
*/
export const DEFAULT_EXCLUDED: ReadonlySet<string> = new Set([
'set_secret',
Expand All @@ -152,6 +157,10 @@ export const DEFAULT_EXCLUDED: ReadonlySet<string> = new Set([
'list_secrets',
'delete_secret',
'get_secrets_public_key',
'add_oidc_trust',
'list_oidc_trust',
'drop_oidc_trust',
'exchange_oidc_token',
]);

/**
Expand Down
52 changes: 52 additions & 0 deletions json/systemSchema.json
Original file line number Diff line number Diff line change
Expand Up @@ -472,5 +472,57 @@
"attribute": "__updatedtime__"
}
]
},
"hdb_oidc_trust": {
"hash_attribute": "id",
"name": "hdb_oidc_trust",
"schema": "system",
"audit": true,
"attributes": [
{
"attribute": "id"
},
{
"attribute": "issuer"
},
{
"attribute": "audience"
},
{
"attribute": "claims"
},
{
"attribute": "user"
},
{
"attribute": "operations"
},
{
"attribute": "enabled"
},
{
"attribute": "description"
},
{
"attribute": "updated_by"
},
{
"attribute": "__createdtime__"
},
{
"attribute": "__updatedtime__"
}
]
},
"hdb_oidc_token_use": {
"hash_attribute": "id",
"name": "hdb_oidc_token_use",
"schema": "system",
"audit": true,
"attributes": [
{
"attribute": "id"
}
]
}
}
Loading
Loading