From 3db8707880604fb31d8825ec06c515318de5f7c2 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 25 Aug 2026 10:42:51 -0400 Subject: [PATCH 1/4] docs(deploy): by-reference deploys and sealed credentials (v5.2.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Splits the shipped half of #599 out so it can merge on its own. Everything here documents behavior in **v5.2.3** — `git tag --contains` on harper#1850's and harper#1851's merge commits returns v5.2.3 and v5.2.4 — while #599 is gated on harper#1849, still a draft. Accurate docs for a shipped feature should not wait on unshipped work. Carries the corrected text, not #599's original: the corrections that lived in #617 are folded in here, so there is no window where the published page describes the superseded parameter form. - **Deploying by Reference** — `harper deploy by_ref=true`, `ref=`, `credential=true`. A reference pins to a resolved SHA: refs resolve from the local checkout, then from the remote when it is absent (a shallow CI clone usually has neither), and the deploy fails closed if neither can name a commit, because peers resolve the package independently. A ref must also name something a clone can fetch — `refs/heads/*` or `refs/tags/*` — so `refs/pull/123/head` is rejected up front even where the local checkout resolves it. A pinned SHA fixes the source revision, not the built artifact. - **`credential=true`** derives the host from the package, so the credential always matches the clone it authenticates; an explicitly named host that does not match is rejected rather than silently unused. - **GitHub Actions behavior** — the unpushed-commit check is skipped there (the runner's checkout is not a branch `git branch -r --contains` can see) while the dirty-tree warning still applies, and a `pull_request` run deploys the PR's head commit rather than the merge commit the runner checks out. - **Provisioning a Deploy Credential** — `harper deploy setup=true`, the client-side sealing flow, the two providers it supports, and that it needs super_user because it calls `get_secrets_public_key` and `set_secret`. - `reference/security/secrets.md` points its private-source section at the CLI flow, and `5.2.md` gains the two release-note entries. The `revert_component` cross-reference from the credential section is deliberately left on #599: `revert_component` is unshipped, and the build caught the dangling anchor when it came across. --- reference/components/applications.md | 90 ++++++++++++++++++++++++++++ reference/security/secrets.md | 2 + release-notes/v5-lincoln/5.2.md | 12 ++++ 3 files changed, 104 insertions(+) diff --git a/reference/components/applications.md b/reference/components/applications.md index a001e3450..5dc7f2ad0 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -136,6 +136,96 @@ Harper generates a `package.json` from component configurations and uses a form For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to register keys first. +### Deploying by Reference + + + +Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. + +Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. + +A pinned SHA fixes the _source_, not the built artifact. The cluster installs and builds from that source on each node, so unpinned dependency ranges, a mutable registry artifact, install scripts, or a different toolchain can still produce different bytes — or a failure — from the same commit. Commit your lockfile if you need the build itself to be reproducible. + +`harper deploy by_ref=true` builds that reference from the local git repository, so you don't assemble the URL yourself: + +```sh +harper deploy by_ref=true restart=true replicated=true +``` + +This resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com//.git#`. + +**Parameters**: + +- `by_ref` - Build the package reference from the local repository. +- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Resolved to a commit SHA before it ships. Implies `by_ref`. +- `credential` _(optional)_ - Set to `true` to authenticate the clone with the stored credential for the repository's host. Omit for public repositories. + +```sh +# Deploy a specific tag +harper deploy ref=v1.2.0 restart=true replicated=true + +# Roll back by deploying an older commit +harper deploy ref=9f8c2a1 restart=true replicated=true +``` + +**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved to a full commit SHA before the deploy is sent — from your local checkout when it has the ref, and from the remote when it doesn't (a shallow CI clone usually doesn't). Annotated tags resolve to the commit they point at. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. + +If a `ref` can't be resolved either way, the deploy stops rather than sending the name for the cluster to resolve. Run `git fetch` and retry, or pass a full commit SHA — that needs no resolution and is always accepted. + +A `ref` must also name something a clone can fetch: `refs/heads/*` and `refs/tags/*`, or a bare branch or tag name. Anything else — `refs/pull/123/head`, say — is rejected up front, even if your own checkout can resolve it, because the cluster could resolve that commit and still never check it out. + +**Commit and push first.** The cluster clones from the remote, so it only sees commits that have been pushed. `by_ref` warns in both directions: when the working tree is dirty (those changes won't be part of the deploy) and when the commit being deployed isn't on any remote branch (the cluster won't be able to clone it). The second check reads your local remote-tracking refs, so run `git fetch` if you get it for a commit you know you pushed. + +The unpushed-commit check is **skipped under GitHub Actions**, where the runner's checkout is not a branch a `git branch -r --contains` can see; the dirty-tree warning still applies. On a `pull_request` run the commit is resolved from the event payload instead, as described below. + +**In GitHub Actions**, `by_ref` deploys the commit the workflow is running on. On a `pull_request` run that is the pull request's **head** commit rather than the merge commit the runner checks out: the merge commit lives under `refs/pull//merge`, which a plain clone can't fetch, so the cluster would have no way to resolve it. For a pull request from a fork, the head repository is the fork, and the CLI names it before deploying. If the event payload isn't readable, the deploy stops and asks for the commit explicitly: + +```sh +harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true +``` + +#### Private repositories + +Pass `credential=true` for a private repository. The CLI attaches a `credentials` reference naming a secret that the cluster resolves in memory at clone time, so no token travels in the operation body or lands on disk: + +```sh +harper deploy by_ref=true credential=true restart=true replicated=true +``` + +The host comes from the package being deployed, so the credential always matches the clone it authenticates. Naming the host explicitly (`credential=github.com`) still works, but one that doesn't match the package's host is rejected instead of deployed — the clone would never ask for it, and the deploy would fail as though no credential were configured. + +Provision that credential once with [`harper deploy setup=true`](#provisioning-a-deploy-credential). See [Private-source deploy credentials](../security/secrets.md#private-source-deploy-credentials) for how the secret is named and resolved, and [`add_ssh_key`](#add_ssh_key) for the SSH-key alternative. + +:::note +Deploying by reference means the **cluster** installs and builds the component from source. If your application needs a build step that can't run on the node, keep shipping the built output as a payload deploy instead. +::: + +### Provisioning a Deploy Credential + + + +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source. It calls `get_secrets_public_key` and `set_secret`, both of which require **super_user**, so run it with an administrative credential rather than the CI identity it provisions for: + +```sh +harper deploy setup=true +``` + +It asks which private source needs a credential (a GitHub repository or an npm registry), sources a token, and then: + +1. Fetches the cluster's public key with `get_secrets_public_key`. +2. **Encrypts the token locally** into an `enc:v1:` envelope. +3. Stores only the ciphertext with `set_secret`, in the component-scoped tier. +4. Grants this component permission to resolve it with `grant_secret`. +5. Prints the `credentials` reference for the deploy to use. + +The plaintext never leaves your machine: the operations API, its logs, and replication only ever carry the envelope, and the cluster decrypts it in memory at deploy time. This requires a cluster with secrets custody (Harper Pro / Fabric) — see [Client-side encryption](../security/secrets.md#client-side-encryption-encrypt-before-it-leaves-the-client). + +**Prefer a fine-grained PAT.** For a GitHub repository the prompt offers, and defaults to, pasting a fine-grained personal access token with **Contents: Read-only on that one repository**. If you have the `gh` CLI authenticated it also offers its session token, which is one keypress cheaper but typically carries `repo`, `read:org`, `gist`, and `workflow` scopes across your whole account; choosing it prints a warning. What this flow seals is durable and replayed on every cold deploy and rollback, so it is worth being the narrowest credential that does the job. + +The secret is stored **scoped to the component**, never in the global `processEnv` tier that every component and child process can read. If a global secret already exists at the derived name, it is converted to the scoped tier — the name is derived from the component, so a global secret there was never serving anything the scoped one doesn't. Existing grants on the row are preserved. + +Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. + ## Dependency Management Harper uses `npm` and `package.json` for dependency management. diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 2e2a0a842..1394b8993 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -289,6 +289,8 @@ function encryptSecret(plaintext, publicKeyPem, kid) { `deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). +To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.2.3+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). + ## Threat model **Protects against:** theft of on-disk config/`.env` files, the editor/operations read surface, secrets appearing in operations logs and replication payloads, and an operator observing traffic at the TLS-terminating layer. Client-side encryption additionally keeps plaintext off the operations API entirely. diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index c2b5c6b94..9fc735132 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -40,6 +40,18 @@ The legacy `allowRead`, `allowUpdate`, `allowCreate`, and `allowDelete` hooks re Components can now declare recurring jobs in their configuration with a new built-in `scheduler` plugin. Jobs run on a five-field cron expression or a simple interval (`90s`, `5m`, `1h`), invoking a designated export from the component. In a cluster, execution is leader-coordinated - under normal operation each occurrence runs once, on an automatically elected leader node - with heartbeat-based failover, catch-up for missed occurrences, and per-job run state recorded in a replicated system table (handlers should be idempotent, as failover can occasionally deliver an occurrence twice; conversely, catch-up only backfills the single most recent missed occurrence, not a full backlog). See [Scheduler](/reference/v5/components/scheduler). +### Deploying by Git Reference (5.2.3) + +`harper deploy by_ref=true` builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. + +A pinned SHA fixes the source revision rather than the built artifact, since the cluster still builds from source on each node. Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a `pull_request` run it deploys the pull request's head commit, not the merge commit the runner checks out. See [Deploying by Reference](/reference/v5/components/applications#deploying-by-reference). + +### Sealed Deploy Credentials (5.2.3) + +`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles the two private-source kinds it supports — a private GitHub repository and a private npm registry — and prints the `credentials` entry the deploy should use. For a git-by-reference deploy that entry is attached by `harper deploy credential=true`, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the `registry` rather than a host. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). + +## Configuration + ## Configuration ### Replicated `set_configuration` From 1fc262e4fdf0fcd2017d71ba4a49924e1553c775 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 25 Aug 2026 11:04:40 -0400 Subject: [PATCH 2/4] docs(security): add_ssh_key server-side keygen (v5.2.4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same reasoning as the by_ref content: harper-pro#594 merged 2026-08-20 and `git tag --contains` puts it in **v5.2.4**, so it is shipped and should not wait on harper#1849. - `generate: true` mints an ed25519 keypair on the node and returns only the public half, so a deploy key's private half never travels in a request body, shell history, or CI log. Generation is in-process (`node:crypto`), needing no `ssh-keygen` binary on the host. - `public_key` comes back only on the generating call — Harper does not retain it — so the recovery path is delete-and-regenerate, since `update_ssh_key` cannot mint. - The encryption-at-rest claim is qualified rather than absolute. `sealSSHKey` falls back to storing and replicating the private key in **plaintext** with only a WARN when no secret custody is registered, and that fallback is deliberate — SSH keys predate custody. So it is a property of the deployment's configuration, not of the operation, which matters most for `generate: true` because that reads as though the key could never be exposed. Custody is present by default, and the note says so rather than implying the risk is likely. Also restores the 5.2.4 release-note entry, which I dropped when I took main's `5.2.md` wholesale during the #599 merge — the same whole-file resolution hazard that cost content once already in this session. Verifying the file matched main confirmed the take succeeded but said nothing about what #599 uniquely added. --- reference/operations-api/operations.md | 47 +++++++++++++++++++++++++- release-notes/v5-lincoln/5.2.md | 4 +++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index db0d3df03..9d282b3c8 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -769,7 +769,19 @@ The deployment must be in a terminal status (`success`, `failed`, or `rolled_bac ### `add_ssh_key` -Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. +Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. + +`list_ssh_keys` and the logs never return key material. + +The stored private key is encrypted at rest and crosses the cluster as ciphertext **when secret custody is configured**. Custody is present by default — the file tier generates a cluster keypair on first boot — so this is the normal case. + +:::warning +On a node with **no** secret custody registered, `add_ssh_key` stores and replicates the private key in **plaintext**. It logs a WARN saying so and the operation still succeeds, because SSH keys predate custody and must keep working on a node that has none. + +That means encryption at rest is a property of your configuration, not a guarantee of the operation. If you are relying on it — and `generate: true` in particular reads as though the key can never be exposed — verify `secretCustody` is configured on every node in the cluster, and check the logs for that warning after adding a key. See [Secrets](../security/secrets.md). +::: + +Adding an existing key: ```json { @@ -781,6 +793,39 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re } ``` +#### Server-side key generation (`generate`) + + + +With `generate: true`, Harper mints an ed25519 keypair on the node handling the request and returns only the **public** half. The private key is created inside the cluster and never travels from a client, so it can't be captured in a shell history, CI log, or request body on the way in: + +```json +{ + "operation": "add_ssh_key", + "name": "my-key", + "generate": true, + "host": "my-key.github.com", + "hostname": "github.com" +} +``` + +Response: + +```json +{ + "message": "Added ssh key: my-key", + "public_key": "ssh-ed25519 AAAAC3Nza... harper:my-key" +} +``` + +Register that `public_key` with your git host (e.g. as a GitHub deploy key) to authorize the deploy. The generated key is commented `harper:` so it's identifiable in the host's key list. + +`key` and `generate` are mutually exclusive — sending both is rejected. Generation happens in-process, so it requires no `ssh-keygen` binary on the host and the minted private key is never written to a temporary file on its way into storage. + +:::note +`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the private key (sealed, subject to the custody caveat above) and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. +::: + --- ## Secrets diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 9fc735132..819b03563 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -91,3 +91,7 @@ Components can also pass `host` and `urlPath` directly to `server.http()`, `serv ### Web Application Firewall Harper Pro now includes a Web Application Firewall that evaluates rule-based IP/CIDR, method, path, header, and query conditions before authentication and application routing. Rules support block, log, and score actions; cluster-wide monitor and off modes; per-rule shadowing; node activation gates; live replicated updates; and RE2-backed regular expressions. See [Web Application Firewall](/reference/v5/web-application-firewall/overview). + +### Server-Side SSH Key Generation (5.2.4) + +`add_ssh_key` accepts `generate: true` to have Harper mint an ed25519 keypair itself and return only the public half, so a deploy key's private half is never carried in a request body, shell history, or CI log. Generation happens in process, requiring no `ssh-keygen` binary on the host. The public key is returned only on the generating call — Harper does not retain it — and is commented `harper:` so it is identifiable in the git host's key list. See [Server-side key generation](/reference/v5/operations-api/operations#server-side-key-generation-generate). From 8de3ed99c98f7684c5ae899797427e476cb53140 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 25 Aug 2026 11:10:54 -0400 Subject: [PATCH 3/4] docs: drop a duplicated header and clarify "before it ships" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two of gemini's three findings on #636. The duplicate `## Configuration` in `5.2.md` is mine: extracting the two 5.2.3 release-note entries by line range swept up the following section heading with them, so the merged file carried it twice. Removed. Extracting by heading boundary rather than line number would have avoided it — the same lesson as the `add_ssh_key` extraction earlier, where an off-by-one line range grabbed a neighbouring section. "Resolved to a commit SHA before it ships" is genuinely ambiguous — "ships" reads as a software release in a paragraph that is otherwise about git refs. Now "before it is sent to the cluster", which is what actually happens. Declining the third: it asks for `` in place of the inline `(v5.2.3+)` in `secrets.md`. That text sits mid-sentence in prose, and CONTRIBUTING.md is explicit — "using the component mid-sentence is awkward. Reserve `` for standalone placement after headings." The exact patch version it also asks for is already there. --- reference/components/applications.md | 2 +- release-notes/v5-lincoln/5.2.md | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/reference/components/applications.md b/reference/components/applications.md index 5dc7f2ad0..071538995 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -157,7 +157,7 @@ This resolves the repository's `origin` remote and the current commit, then depl **Parameters**: - `by_ref` - Build the package reference from the local repository. -- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Resolved to a commit SHA before it ships. Implies `by_ref`. +- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Resolved to a commit SHA before it is sent to the cluster. Implies `by_ref`. - `credential` _(optional)_ - Set to `true` to authenticate the clone with the stored credential for the repository's host. Omit for public repositories. ```sh diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index 819b03563..40341b4d0 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -52,8 +52,6 @@ A pinned SHA fixes the source revision rather than the built artifact, since the ## Configuration -## Configuration - ### Replicated `set_configuration` The `set_configuration` operation now accepts `"replicated": true` to apply a configuration change to all cluster nodes in a single Operations API call, with per-node outcomes reported in the response's `replicated` array. Only cluster-appropriate parameters should be replicated — see [Configuration Operations](/reference/v5/configuration/operations#set-configuration). From e38eaff41eb2bf67138110ec2d10d87734f142b6 Mon Sep 17 00:00:00 2001 From: Dawson Toth Date: Tue, 25 Aug 2026 12:08:20 -0400 Subject: [PATCH 4/4] docs(deploy): complete the super_user call list for deploy setup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per @cb1kenobi: the flow also calls `grant_secret` (bin/deploySetup.ts:130, step 5 of its header comment), which I omitted. It is equally SU-only (utility/operation_authorization.ts:345), so the guidance was right and only the list was short. Worth noting the omission is not random — my own text three paragraphs above describes the flow as five steps including grant_secret, so the call list contradicted the flow description on the same page. --- reference/components/applications.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/components/applications.md b/reference/components/applications.md index 071538995..b1c6a0ac1 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -204,7 +204,7 @@ Deploying by reference means the **cluster** installs and builds the component f -`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source. It calls `get_secrets_public_key` and `set_secret`, both of which require **super_user**, so run it with an administrative credential rather than the CI identity it provisions for: +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source. It calls `get_secrets_public_key`, `set_secret`, and `grant_secret`, all of which require **super_user**, so run it with an administrative credential rather than the CI identity it provisions for: ```sh harper deploy setup=true