diff --git a/reference/components/applications.md b/reference/components/applications.md index a001e345..b1c6a0ac 100644 --- a/reference/components/applications.md +++ b/reference/components/applications.md @@ -136,6 +136,96 @@ Harper generates a `package.json` from component configurations and uses a form For SSH-based private repos, use the [Add SSH Key](#add_ssh_key) operation to register keys first. +### Deploying by Reference + + + +Omitting `package` uploads a snapshot of your working directory. The result is an anonymous artifact: nothing records _which_ commit it came from, so reproducing it later — or stepping back to a previous release — means finding those exact files again. + +Deploying by **reference** sends a pinned git reference instead, and the cluster fetches that exact commit. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. + +A pinned SHA fixes the _source_, not the built artifact. The cluster installs and builds from that source on each node, so unpinned dependency ranges, a mutable registry artifact, install scripts, or a different toolchain can still produce different bytes — or a failure — from the same commit. Commit your lockfile if you need the build itself to be reproducible. + +`harper deploy by_ref=true` builds that reference from the local git repository, so you don't assemble the URL yourself: + +```sh +harper deploy by_ref=true restart=true replicated=true +``` + +This resolves the repository's `origin` remote and the current commit, then deploys `package=git+https://github.com//.git#`. + +**Parameters**: + +- `by_ref` - Build the package reference from the local repository. +- `ref` _(optional)_ - Deploy a specific commit, tag, or branch instead of `HEAD`. Resolved to a commit SHA before it is sent to the cluster. Implies `by_ref`. +- `credential` _(optional)_ - Set to `true` to authenticate the clone with the stored credential for the repository's host. Omit for public repositories. + +```sh +# Deploy a specific tag +harper deploy ref=v1.2.0 restart=true replicated=true + +# Roll back by deploying an older commit +harper deploy ref=9f8c2a1 restart=true replicated=true +``` + +**A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved to a full commit SHA before the deploy is sent — from your local checkout when it has the ref, and from the remote when it doesn't (a shallow CI clone usually doesn't). Annotated tags resolve to the commit they point at. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. + +If a `ref` can't be resolved either way, the deploy stops rather than sending the name for the cluster to resolve. Run `git fetch` and retry, or pass a full commit SHA — that needs no resolution and is always accepted. + +A `ref` must also name something a clone can fetch: `refs/heads/*` and `refs/tags/*`, or a bare branch or tag name. Anything else — `refs/pull/123/head`, say — is rejected up front, even if your own checkout can resolve it, because the cluster could resolve that commit and still never check it out. + +**Commit and push first.** The cluster clones from the remote, so it only sees commits that have been pushed. `by_ref` warns in both directions: when the working tree is dirty (those changes won't be part of the deploy) and when the commit being deployed isn't on any remote branch (the cluster won't be able to clone it). The second check reads your local remote-tracking refs, so run `git fetch` if you get it for a commit you know you pushed. + +The unpushed-commit check is **skipped under GitHub Actions**, where the runner's checkout is not a branch a `git branch -r --contains` can see; the dirty-tree warning still applies. On a `pull_request` run the commit is resolved from the event payload instead, as described below. + +**In GitHub Actions**, `by_ref` deploys the commit the workflow is running on. On a `pull_request` run that is the pull request's **head** commit rather than the merge commit the runner checks out: the merge commit lives under `refs/pull//merge`, which a plain clone can't fetch, so the cluster would have no way to resolve it. For a pull request from a fork, the head repository is the fork, and the CLI names it before deploying. If the event payload isn't readable, the deploy stops and asks for the commit explicitly: + +```sh +harper deploy ref=${{ github.event.pull_request.head.sha }} restart=true replicated=true +``` + +#### Private repositories + +Pass `credential=true` for a private repository. The CLI attaches a `credentials` reference naming a secret that the cluster resolves in memory at clone time, so no token travels in the operation body or lands on disk: + +```sh +harper deploy by_ref=true credential=true restart=true replicated=true +``` + +The host comes from the package being deployed, so the credential always matches the clone it authenticates. Naming the host explicitly (`credential=github.com`) still works, but one that doesn't match the package's host is rejected instead of deployed — the clone would never ask for it, and the deploy would fail as though no credential were configured. + +Provision that credential once with [`harper deploy setup=true`](#provisioning-a-deploy-credential). See [Private-source deploy credentials](../security/secrets.md#private-source-deploy-credentials) for how the secret is named and resolved, and [`add_ssh_key`](#add_ssh_key) for the SSH-key alternative. + +:::note +Deploying by reference means the **cluster** installs and builds the component from source. If your application needs a build step that can't run on the node, keep shipping the built output as a payload deploy instead. +::: + +### Provisioning a Deploy Credential + + + +`harper deploy setup=true` provisions the credential a private deploy needs. It's interactive, and runs once per component and source. It calls `get_secrets_public_key`, `set_secret`, and `grant_secret`, all of which require **super_user**, so run it with an administrative credential rather than the CI identity it provisions for: + +```sh +harper deploy setup=true +``` + +It asks which private source needs a credential (a GitHub repository or an npm registry), sources a token, and then: + +1. Fetches the cluster's public key with `get_secrets_public_key`. +2. **Encrypts the token locally** into an `enc:v1:` envelope. +3. Stores only the ciphertext with `set_secret`, in the component-scoped tier. +4. Grants this component permission to resolve it with `grant_secret`. +5. Prints the `credentials` reference for the deploy to use. + +The plaintext never leaves your machine: the operations API, its logs, and replication only ever carry the envelope, and the cluster decrypts it in memory at deploy time. This requires a cluster with secrets custody (Harper Pro / Fabric) — see [Client-side encryption](../security/secrets.md#client-side-encryption-encrypt-before-it-leaves-the-client). + +**Prefer a fine-grained PAT.** For a GitHub repository the prompt offers, and defaults to, pasting a fine-grained personal access token with **Contents: Read-only on that one repository**. If you have the `gh` CLI authenticated it also offers its session token, which is one keypress cheaper but typically carries `repo`, `read:org`, `gist`, and `workflow` scopes across your whole account; choosing it prints a warning. What this flow seals is durable and replayed on every cold deploy and rollback, so it is worth being the narrowest credential that does the job. + +The secret is stored **scoped to the component**, never in the global `processEnv` tier that every component and child process can read. If a global secret already exists at the derived name, it is converted to the scoped tier — the name is derived from the component, so a global secret there was never serving anything the scoped one doesn't. Existing grants on the row are preserved. + +Because the stored token is durable, later deploys — including re-fetching an older reference — reuse it without re-entering anything. + ## Dependency Management Harper uses `npm` and `package.json` for dependency management. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index db0d3df0..9d282b3c 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -769,7 +769,19 @@ The deployment must be in a terminal status (`success`, `failed`, or `rolled_bac ### `add_ssh_key` -Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. +Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. + +`list_ssh_keys` and the logs never return key material. + +The stored private key is encrypted at rest and crosses the cluster as ciphertext **when secret custody is configured**. Custody is present by default — the file tier generates a cluster keypair on first boot — so this is the normal case. + +:::warning +On a node with **no** secret custody registered, `add_ssh_key` stores and replicates the private key in **plaintext**. It logs a WARN saying so and the operation still succeeds, because SSH keys predate custody and must keep working on a node that has none. + +That means encryption at rest is a property of your configuration, not a guarantee of the operation. If you are relying on it — and `generate: true` in particular reads as though the key can never be exposed — verify `secretCustody` is configured on every node in the cluster, and check the logs for that warning after adding a key. See [Secrets](../security/secrets.md). +::: + +Adding an existing key: ```json { @@ -781,6 +793,39 @@ Adds an SSH key (must be ed25519) for authenticating deployments from private re } ``` +#### Server-side key generation (`generate`) + + + +With `generate: true`, Harper mints an ed25519 keypair on the node handling the request and returns only the **public** half. The private key is created inside the cluster and never travels from a client, so it can't be captured in a shell history, CI log, or request body on the way in: + +```json +{ + "operation": "add_ssh_key", + "name": "my-key", + "generate": true, + "host": "my-key.github.com", + "hostname": "github.com" +} +``` + +Response: + +```json +{ + "message": "Added ssh key: my-key", + "public_key": "ssh-ed25519 AAAAC3Nza... harper:my-key" +} +``` + +Register that `public_key` with your git host (e.g. as a GitHub deploy key) to authorize the deploy. The generated key is commented `harper:` so it's identifiable in the host's key list. + +`key` and `generate` are mutually exclusive — sending both is rejected. Generation happens in-process, so it requires no `ssh-keygen` binary on the host and the minted private key is never written to a temporary file on its way into storage. + +:::note +`public_key` is returned **only** on the generating call — that response is the one time the public half is handed back. Harper stores the private key (sealed, subject to the custody caveat above) and the host config; it does not retain the public key for later retrieval, and `update_ssh_key` requires a key you supply (it can't mint one). So capture `public_key` from this response — if you lose it, `delete_ssh_key` then `add_ssh_key` with `generate: true` again to mint a fresh pair, and re-register the new public key with your git host. +::: + --- ## Secrets diff --git a/reference/security/secrets.md b/reference/security/secrets.md index 2e2a0a84..1394b899 100644 --- a/reference/security/secrets.md +++ b/reference/security/secrets.md @@ -289,6 +289,8 @@ function encryptSecret(plaintext, publicKeyPem, kid) { `deploy_component` accepts a `credentials` array so a component installed from a private **npm registry** or private **git repository** can authenticate. A provided token is ingested into the secrets store (as a reference, encrypted) rather than travelling in the operation body, persisting as a plaintext `.npmrc`, or being written to disk for git — so package-reference deploys survive rollback, reboot, and new peers joining. Ingested tokens are stored under a derived name (`deploy..` or `deploy..git.`) granted to the component. See [`deploy_component`](../operations-api/operations.md#deploy_component). +To provision one of these without handing the cluster a plaintext token at all, `harper deploy setup=true` (v5.2.3+) runs the [client-side encryption](#client-side-encryption-encrypt-before-it-leaves-the-client) flow above for you: it fetches the public key, seals the token locally into an `enc:v1:` envelope, stores only the ciphertext under that same derived name, and prints the `credentials` reference for the deploy to use. See [Provisioning a Deploy Credential](../components/applications.md#provisioning-a-deploy-credential). + ## Threat model **Protects against:** theft of on-disk config/`.env` files, the editor/operations read surface, secrets appearing in operations logs and replication payloads, and an operator observing traffic at the TLS-terminating layer. Client-side encryption additionally keeps plaintext off the operations API entirely. diff --git a/release-notes/v5-lincoln/5.2.md b/release-notes/v5-lincoln/5.2.md index c2b5c6b9..40341b4d 100644 --- a/release-notes/v5-lincoln/5.2.md +++ b/release-notes/v5-lincoln/5.2.md @@ -40,6 +40,16 @@ The legacy `allowRead`, `allowUpdate`, `allowCreate`, and `allowDelete` hooks re Components can now declare recurring jobs in their configuration with a new built-in `scheduler` plugin. Jobs run on a five-field cron expression or a simple interval (`90s`, `5m`, `1h`), invoking a designated export from the component. In a cluster, execution is leader-coordinated - under normal operation each occurrence runs once, on an automatically elected leader node - with heartbeat-based failover, catch-up for missed occurrences, and per-job run state recorded in a replicated system table (handlers should be idempotent, as failover can occasionally deliver an occurrence twice; conversely, catch-up only backfills the single most recent missed occurrence, not a full backlog). See [Scheduler](/reference/v5/components/scheduler). +### Deploying by Git Reference (5.2.3) + +`harper deploy by_ref=true` builds a package reference from the local git repository, so a deploy records which commit it came from instead of shipping an anonymous snapshot of the working directory. Redeploying the same reference deploys the same source revision, and rolling back is deploying an older one. + +A pinned SHA fixes the source revision rather than the built artifact, since the cluster still builds from source on each node. Tags and branches are resolved to a full commit SHA before the deploy is sent — from the local checkout when it has the ref, and from the remote when it doesn't, as a shallow CI clone usually doesn't. A ref that can't be resolved either way stops the deploy rather than being sent by name, because peers resolve the package independently and a name that moves mid-deploy would leave nodes on different code. The CLI warns when the working tree is dirty and when the commit isn't on any remote branch. Under GitHub Actions on a `pull_request` run it deploys the pull request's head commit, not the merge commit the runner checks out. See [Deploying by Reference](/reference/v5/components/applications#deploying-by-reference). + +### Sealed Deploy Credentials (5.2.3) + +`harper deploy setup=true` provisions the credential a private-source deploy needs, sealing the token locally before it leaves the machine: the CLI fetches the instance's public key, encrypts the token into an `enc:v1:` envelope, and stores only the ciphertext. The plaintext never reaches the operations API, its logs, or replication. It handles the two private-source kinds it supports — a private GitHub repository and a private npm registry — and prints the `credentials` entry the deploy should use. For a git-by-reference deploy that entry is attached by `harper deploy credential=true`, which derives the host from the package so the credential always matches the clone it authenticates; an npm-registry credential is attached by passing the printed entry, which names the `registry` rather than a host. See [Provisioning a Deploy Credential](/reference/v5/components/applications#provisioning-a-deploy-credential). + ## Configuration ### Replicated `set_configuration` @@ -79,3 +89,7 @@ Components can also pass `host` and `urlPath` directly to `server.http()`, `serv ### Web Application Firewall Harper Pro now includes a Web Application Firewall that evaluates rule-based IP/CIDR, method, path, header, and query conditions before authentication and application routing. Rules support block, log, and score actions; cluster-wide monitor and off modes; per-rule shadowing; node activation gates; live replicated updates; and RE2-backed regular expressions. See [Web Application Firewall](/reference/v5/web-application-firewall/overview). + +### Server-Side SSH Key Generation (5.2.4) + +`add_ssh_key` accepts `generate: true` to have Harper mint an ed25519 keypair itself and return only the public half, so a deploy key's private half is never carried in a request body, shell history, or CI log. Generation happens in process, requiring no `ssh-keygen` binary on the host. The public key is returned only on the generating call — Harper does not retain it — and is commented `harper:` so it is identifiable in the git host's key list. See [Server-side key generation](/reference/v5/operations-api/operations#server-side-key-generation-generate).