bozemanpass · dboreham · Aug 4, 2026 · Aug 4, 2026
diff --git a/README.md b/README.md
@@ -2,9 +2,9 @@
 
 Stack allows building and deployment of a system of related containerized applications as a single "stack". Transparently deploy to local Docker, Podman or to remote Kubernetes.
 
-![Building and deploying a three-container todo app — front end, API and PostgreSQL — to local Docker with stack](./docs/images/quickstart.gif)
+![Building a three-container todo app — front end, API and PostgreSQL — and deploying it with stack, first to local Docker and then, unchanged, to a Kubernetes cluster over HTTPS](./docs/images/quickstart.gif)
 
-_An unedited recording of the [Docker quick start](#docker) below. Regenerate it with `./demo/record-quickstart.sh`._
+_An unedited recording of the [Docker quick start](#docker) below, followed by the same stack deployed to a real Kubernetes cluster. Regenerate it with `./demo/k8s-host.sh create && ./demo/record-quickstart.sh` (see [demo/README.md](./demo/README.md))._
 
 ## What is Stack good for?
 Stack is useful for a wide category of software applications including those that have a web app component, back-end services and optionally a database: web systems.

diff --git a/demo/README.md b/demo/README.md
@@ -0,0 +1,85 @@
+# Demo recordings
+
+`docs/images/quickstart.gif`, the animation at the top of the project README, is
+recorded from `quickstart.tape` by [vhs](https://github.com/charmbracelet/vhs).
+Nothing in it is faked or edited: every command really runs, against the
+[example todo stack](https://github.com/bozemanpass/example-todo-list), and it
+deploys that stack twice — first to local Docker, then the same stack, unchanged
+except for its deployment spec, to a real single-node Kubernetes (k3s) cluster
+on a cloud VM, reached over HTTPS with a real Let's Encrypt certificate.
+
+## Recording it
+
+The Kubernetes cluster is provisioned separately from the recording, because
+creating a VM and installing k3s, cert-manager and the rest takes minutes and
+costs money, while getting a take you like usually means recording several
+times:
+
+```bash
+./demo/k8s-host.sh create      # once, ~5 minutes
+./demo/record-quickstart.sh    # as many takes as you like
+./demo/k8s-host.sh destroy     # when you are done for the day
+```
+
+`k8s-host.sh info` prints the current host. The host state (its name, id and
+FQDN, and a kubeconfig that reaches it) lives in `~/.cache/stack-demo/k8s-host`,
+overridable with `STACK_DEMO_STATE_DIR`.
+
+## Requirements
+
+* `vhs`, `ttyd`, `ffmpeg`, `docker`, `jq`, `openssl`, `kubectl`, and `stack` on `PATH`
+  (`sudo apt-get install -y ffmpeg ttyd`; vhs from its
+  [releases page](https://github.com/charmbracelet/vhs/releases))
+* [`machine`](https://github.com/stirlingbridge/machine), for `k8s-host.sh`
+* Environment for the cloud host and the image registry — the same variable
+  names the k3s CI job passes to `tests/k3s-deploy/run-k3s-deploy-test.sh`:
+  `MACHINE_DO_TOKEN`, `MACHINE_SSH_KEY_NAME`, `MACHINE_SSH_KEY_FILE`,
+  `MACHINE_DNS_ZONE`, `MACHINE_PROJECT`, `STACK_IMAGE_REGISTRY`,
+  `STACK_IMAGE_REGISTRY_USER`, `STACK_IMAGE_REGISTRY_TOKEN`,
+  `LETSENCRYPT_EMAIL`. `k8s-host.sh` provisions the host exactly the way that
+  test provisions its cluster, so a working setup for one works for the other.
+
+`record-quickstart.sh` only needs the registry variables and an existing host;
+it never creates or destroys cloud resources.
+
+## How the recording stays short
+
+The animation has to be watchable in a README, so the recorder prepares state
+off camera such that each recorded command still does real work but returns
+quickly: the repo clone happens into a scratch `STACK_REPO_BASE_DIR`, the todo
+image *tags* are dropped while Docker's layer cache is kept (so the recorded
+build is genuine but takes seconds), and the images are pushed to the registry
+in advance so the recorded `push-images` has nothing left to upload.
+
+The waits that cannot be prepared away — the frontend's dev server coming up
+under Docker, and on the cluster the pods becoming ready, the gateway routing
+to them and, on a first deployment of a hostname, the ACME HTTP-01 exchange —
+happen in `Hide` blocks. Each of those blocks ends with a `clear` *inside* the
+block: `Hide` stops recording frames but the terminal still holds the line that
+was typed, so without it the wait loop would sit on screen for the rest of the
+recording.
+
+Two details keep repeated takes cheap:
+
+* The recorded `stack deploy` passes a fixed `--cluster` name. That name is the
+  Kubernetes namespace, and the certificate Secret is named after it, so
+  redeploying reuses the certificate cert-manager already holds instead of
+  asking Let's Encrypt for a new one every take (see `remove_https_listener` in
+  `src/stack/deploy/k8s/gateway.py`).
+* The VM's node image cache and the registry both survive between takes.
+
+## The tape is a template
+
+The Kubernetes host's FQDN, its kubeconfig path, the image registry and the
+host ports the Docker half curls are all decided outside the tape, so
+`quickstart.tape` carries them as `@@...@@` placeholders.
+`record-quickstart.sh` substitutes them into a rendered copy under
+`/tmp/stack-demo` and runs vhs against that. Run vhs against the rendered copy,
+not against `quickstart.tape` itself.
+
+The ports are worked out by generating a throwaway Compose spec and reading the
+mapped host ports out of it, rather than being written down anywhere — the todo
+stack has moved its frontend and backend ports before, and a recording that
+curls a stale port either prints a failure or, worse, quietly gets an answer
+from something else. For the same reason the recorder refuses to start if
+anything is already listening on one of those ports.
diff --git a/demo/k8s-host.sh b/demo/k8s-host.sh
@@ -0,0 +1,297 @@
+#!/usr/bin/env bash
+#
+# Create, inspect and destroy the Kubernetes host used by the quick-start
+# recording.
+#
+# The recording (demo/quickstart.tape) deploys the same stack twice: once to
+# local Docker, then to a real single-node k3s cluster with a real hostname and
+# a real Let's Encrypt certificate. That cluster lives on a cloud VM, which
+# takes several minutes and real money to provision -- far too slow to do on
+# every take. So the VM's lifecycle is separate from the recording's:
+#
+#     ./demo/k8s-host.sh create      # once, ~5 minutes
+#     ./demo/record-quickstart.sh    # as many takes as you like
+#     ./demo/k8s-host.sh destroy     # when you are done for the day
+#
+# The VM is provisioned exactly the way the k3s CI job provisions its test
+# cluster (tests/k3s-deploy/run-k3s-deploy-test.sh) -- the same "machine"
+# utility, the same machine-provisioning scripts, the same environment variable
+# names -- so a host that works for one works for the other.
+#
+# State (the machine's name, id and FQDN, and a kubeconfig that reaches it) is
+# written to $STACK_DEMO_STATE_DIR, where record-quickstart.sh picks it up.
+#
+# Requires: machine (https://github.com/stirlingbridge/machine), jq, ssh.
+#
+# Required environment (same names the k3s test takes):
+#   MACHINE_DO_TOKEN         DigitalOcean API token
+#   MACHINE_SSH_KEY_NAME     Name of an SSH key registered at the provider
+#   MACHINE_SSH_KEY_FILE     Path to the matching private key file
+#   MACHINE_DNS_ZONE         DNS zone hosted at the provider; the demo hostname
+#                            is a name under this zone
+#   STACK_IMAGE_REGISTRY     Registry the demo's images are pushed to and the
+#                            cluster pulls them from, including any org path
+#   STACK_IMAGE_REGISTRY_USER
+#   STACK_IMAGE_REGISTRY_TOKEN
+#                            Credentials for that registry (for a DigitalOcean
+#                            registry, pass the API token as both)
+#   LETSENCRYPT_EMAIL        Let's Encrypt contact address
+#
+# Optional environment:
+#   STACK_DEMO_STATE_DIR     Where to keep the host state
+#                            (default ~/.cache/stack-demo/k8s-host)
+#   STACK_DEMO_MACHINE_NAME  Machine hostname under the zone (default stackdemo)
+#   MACHINE_REGION           Provider region (default nyc3)
+#   MACHINE_SIZE             Machine size slug (default s-2vcpu-4gb)
+#   MACHINE_IMAGE            Machine image (default ubuntu-24-04-x64)
+#   MACHINE_PROJECT          DigitalOcean project to assign the VM to
+#   MACHINE_PROVISIONING_URL URL of combine.sh (default: the main branch)
+#   MACHINE_CMD              The machine command to run (default: machine)
+#
+set -e
+
+if [ -n "$STACK_SCRIPT_DEBUG" ]; then
+  set -x
+fi
+
+MACHINE_CMD=${MACHINE_CMD:-machine}
+MACHINE_REGION=${MACHINE_REGION:-nyc3}
+MACHINE_SIZE=${MACHINE_SIZE:-s-2vcpu-4gb}
+MACHINE_IMAGE=${MACHINE_IMAGE:-ubuntu-24-04-x64}
+MACHINE_PROVISIONING_URL=${MACHINE_PROVISIONING_URL:-https://raw.githubusercontent.com/stirlingbridge/machine-provisioning/refs/heads/main/scripts/combine.sh}
+MACHINE_NEW_USER=stackdemo
+
+STATE_DIR=${STACK_DEMO_STATE_DIR:-$HOME/.cache/stack-demo/k8s-host}
+MACHINE_NAME=${STACK_DEMO_MACHINE_NAME:-stackdemo}
+
+# The demo deploys to the Gateway API arrangement (k3s-node.sh's own default),
+# which is what gets the application its own Let's Encrypt certificate.
+machine_config=$STATE_DIR/machine-config.yml
+host_env=$STATE_DIR/host.env
+kube_config=$STATE_DIR/kubeconfig
+
+# How long to allow for the VM to boot and cloud-init to install k3s,
+# cert-manager etc. (seconds).
+PROVISION_TIMEOUT=1800
+
+usage () {
+  cat <<'USAGE'
+Usage: demo/k8s-host.sh <command>
+
+  create    Provision the demo's k3s host (no-op if one already exists)
+  info      Print the current host's name, id and FQDN
+  destroy   Destroy the host and delete its DNS record
+USAGE
+}
+
+require_tools () {
+  for cmd in "$MACHINE_CMD" jq ssh; do
+    if ! command -v "$cmd" &> /dev/null; then
+      echo "Error: $cmd is not installed." >&2
+      exit 1
+    fi
+  done
+}
+
+require_env () {
+  missing=""
+  for var in MACHINE_DO_TOKEN MACHINE_SSH_KEY_NAME MACHINE_SSH_KEY_FILE MACHINE_DNS_ZONE \
+             STACK_IMAGE_REGISTRY STACK_IMAGE_REGISTRY_USER STACK_IMAGE_REGISTRY_TOKEN LETSENCRYPT_EMAIL; do
+    if [ -z "${!var}" ]; then
+      missing="$missing $var"
+    fi
+  done
+  if [ -n "$missing" ]; then
+    echo "Error: required environment not set:$missing" >&2
+    exit 1
+  fi
+  if [ ! -f "$MACHINE_SSH_KEY_FILE" ]; then
+    echo "Error: MACHINE_SSH_KEY_FILE $MACHINE_SSH_KEY_FILE does not exist" >&2
+    exit 1
+  fi
+}
+
+write_machine_config () {
+  # The registry credentials in script-args end up in the VM's cloud-init user
+  # data; use a registry and credentials dedicated to demos and testing.
+  # health.sh serves the status endpoint that "machine status" polls;
+  # k3s-node.sh needs the registry credentials so the cluster can pull the
+  # demo's images.
+  mkdir -p "$STATE_DIR"
+  cat > "$machine_config" <<EOF
+digital-ocean:
+    access-token: ${MACHINE_DO_TOKEN}
+    ssh-key: ${MACHINE_SSH_KEY_NAME}
+    dns-zone: ${MACHINE_DNS_ZONE}
+    machine-size: ${MACHINE_SIZE}
+    image: ${MACHINE_IMAGE}
+    region: ${MACHINE_REGION}
+EOF
+  if [ -n "$MACHINE_PROJECT" ]; then
+    echo "    project: ${MACHINE_PROJECT}" >> "$machine_config"
+  fi
+  cat >> "$machine_config" <<EOF
+machines:
+    k8s-stack-host:
+        new-user-name: ${MACHINE_NEW_USER}
+        script-dir: /opt/stackdemo
+        script-url: ${MACHINE_PROVISIONING_URL}
+        script-path: /opt/stackdemo/combine.sh
+        script-args:
+          - health.sh
+          - fqdn.sh
+          - k3s-node.sh -y --letsencrypt-email ${LETSENCRYPT_EMAIL} --image-registry ${STACK_IMAGE_REGISTRY} --image-registry-username ${STACK_IMAGE_REGISTRY_USER} --image-registry-password ${STACK_IMAGE_REGISTRY_TOKEN}
+EOF
+  chmod 600 "$machine_config"
+}
+
+machine_ssh () {
+  ssh -i "$MACHINE_SSH_KEY_FILE" -o StrictHostKeyChecking=accept-new \
+    -o UserKnownHostsFile="$STATE_DIR/known_hosts" "${MACHINE_NEW_USER}@${1}" "$2"
+}
+
+do_create () {
+  require_tools
+  require_env
+
+  machine_fqdn="${MACHINE_NAME}.${MACHINE_DNS_ZONE}"
+
+  if [ -f "$host_env" ]; then
+    # shellcheck disable=SC1090
+    . "$host_env"
+    if [ -f "$machine_config" ] && \
+       [ "$($MACHINE_CMD --config-file "$machine_config" list --id "$STACK_DEMO_MACHINE_ID" --output json 2>/dev/null \
+            | jq -r 'length' 2>/dev/null)" == "1" ]; then
+      echo "Demo k8s host already exists:"
+      do_info
+      echo
+      echo "Destroy it with ./demo/k8s-host.sh destroy, or record with ./demo/record-quickstart.sh"
+      return 0
+    fi
+    echo "Stale host state in $STATE_DIR (no such machine); re-creating"
+    rm -f "$host_env" "$kube_config"
+  fi
+
+  write_machine_config
+
+  echo "Creating machine $machine_fqdn"
+  $MACHINE_CMD --config-file "$machine_config" create --name "$MACHINE_NAME" --type k8s-stack-host --wait-for-ip
+
+  machine_id=$($MACHINE_CMD --config-file "$machine_config" list --name "$MACHINE_NAME" --output json | jq -r '.[0].id')
+  if [ -z "$machine_id" ] || [ "$machine_id" == "null" ]; then
+    echo "Error: could not determine the id of the created machine" >&2
+    exit 1
+  fi
+  echo "Machine created with id $machine_id"
+  # Record the id before provisioning finishes, so a failed or interrupted
+  # provision still leaves something "destroy" can clean up.
+  cat > "$host_env" <<EOF
+STACK_DEMO_MACHINE_NAME=${MACHINE_NAME}
+STACK_DEMO_MACHINE_ID=${machine_id}
+STACK_DEMO_MACHINE_FQDN=${machine_fqdn}
+EOF
+
+  echo "Waiting for provisioning to complete (up to ${PROVISION_TIMEOUT}s)..."
+  start_time=$SECONDS
+  provision_status="UNKNOWN"
+  while [ $((SECONDS - start_time)) -lt $PROVISION_TIMEOUT ]; do
+    provision_status=$($MACHINE_CMD --config-file "$machine_config" status --id "$machine_id" --output json \
+      | jq -r '.[0]["cloud-init-status"]')
+    case "$provision_status" in
+      UP)
+        break
+        ;;
+      ERROR)
+        echo "Provisioning failed; fetching the cloud-init log:" >&2
+        machine_ssh "$machine_fqdn" "sudo tail -100 /var/log/cloud-init-output.log" || true
+        exit 1
+        ;;
+      *)
+        sleep 15
+        ;;
+    esac
+  done
+  if [ "$provision_status" != "UP" ]; then
+    echo "Error: timed out waiting for provisioning to complete (last status: $provision_status)" >&2
+    exit 1
+  fi
+  echo "Provisioning complete"
+
+  # The kubeconfig k3s writes names the local address; the machine's FQDN is in
+  # the API server certificate (k3s-node.sh adds it as a tls-san), so
+  # substituting it yields a kubeconfig that works remotely.
+  machine_ssh "$machine_fqdn" "sudo cat /etc/rancher/k3s/k3s.yaml" \
+    | sed "s/127.0.0.1/${machine_fqdn}/g" > "$kube_config"
+  chmod 600 "$kube_config"
+  if ! grep -q "$machine_fqdn" "$kube_config"; then
+    echo "Error: failed to fetch a usable kubeconfig from the machine" >&2
+    exit 1
+  fi
+  echo "Fetched kubeconfig to $kube_config"
+
+  # The hostname must resolve locally before the recording's HTTPS checks can
+  # pass (the authoritative record was just created; Let's Encrypt resolves it
+  # independently).
+  echo "Waiting for $machine_fqdn to resolve..."
+  for _ in {1..60}; do
+    if getent hosts "$machine_fqdn" > /dev/null; then
+      break
+    fi
+    sleep 5
+  done
+
+  echo
+  echo "Demo k8s host ready:"
+  do_info
+  echo
+  echo "Now record with ./demo/record-quickstart.sh, and destroy the host with"
+  echo "./demo/k8s-host.sh destroy when you are finished."
+}
+
+do_info () {
+  if [ ! -f "$host_env" ]; then
+    echo "No demo k8s host (nothing in $STATE_DIR)."
+    echo "Create one with ./demo/k8s-host.sh create"
+    return 1
+  fi
+  # shellcheck disable=SC1090
+  . "$host_env"
+  echo "  name:       $STACK_DEMO_MACHINE_NAME"
+  echo "  id:         $STACK_DEMO_MACHINE_ID"
+  echo "  fqdn:       $STACK_DEMO_MACHINE_FQDN"
+  echo "  kubeconfig: $kube_config"
+}
+
+do_destroy () {
+  require_tools
+  if [ ! -f "$host_env" ]; then
+    echo "No demo k8s host to destroy (nothing in $STATE_DIR)."
+    return 0
+  fi
+  # shellcheck disable=SC1090
+  . "$host_env"
+  echo "Destroying machine $STACK_DEMO_MACHINE_NAME ($STACK_DEMO_MACHINE_ID)"
+  $MACHINE_CMD --config-file "$machine_config" destroy --no-confirm --delete-dns "$STACK_DEMO_MACHINE_ID"
+  rm -rf "$STATE_DIR"
+  echo "Destroyed"
+}
+
+case "${1:-}" in
+  create)
+    do_create
+    ;;
+  info)
+    do_info
+    ;;
+  destroy)
+    do_destroy
+    ;;
+  ""|-h|--help|help)
+    usage
+    ;;
+  *)
+    echo "Error: unknown command '$1'" >&2
+    usage >&2
+    exit 1
+    ;;
+esac