Document developer flow

README.md

@@ -115,6 +115,7 @@ Detailed documentation on the installation of stack and its prerequisites as wel
 ## Learn More
  - [Stack commands](./docs/commands.md)
  - [Stack files](./docs/stack-files.md)
+ - [Developing an application with stack](./docs/developing-applications.md)
  - [Container wrappers](./docs/wrappers.md)
  - [Building and running webapps](./docs/webapp.md)
  - [Recent New Features](./docs/recent-features.md)

docs/README.md

@@ -36,6 +36,7 @@ stack authors should read [stack-files.md](stack-files.md) first.
 
 | Document | Description |
 | --- | --- |
+| [developing-applications.md](developing-applications.md) | The edit-build-deploy loop for an application you are actively changing: building images from your own working tree and getting each edit into a running compose, kind, or Kubernetes deployment. |
 | [from-laptop-to-production.md](from-laptop-to-production.md) | Choosing a deployment target by situation: local development, a single always-on VM for real users, or Kubernetes — and why a PaaS is not required. Start here if you know your goal but not the options. |
 | [ingress.md](ingress.md) | Automatic HTTP route configuration for an ingress controller / reverse proxy via annotations in `composefile.yml`. |
 | [gateway-api.md](gateway-api.md) | HTTPS on Kubernetes via the Gateway API (with the legacy Ingress API as fallback), and the cluster contract required to use it. |

docs/developing-applications.md

@@ -0,0 +1,210 @@
+# Developing an Application with `stack`
+
+Most of this documentation describes deploying a stack whose containers are built from
+committed, published source. This page covers the other half of the day: you are *editing*
+the application — changing the front end, fixing the API — and you want each edit to show up
+in a running deployment, locally or on a real cluster, without committing and pushing first.
+
+The short version: point `stack` at your own checkout, build with an explicit build policy,
+and know how each deployment target picks up a new image. The examples use
+[example-todo-list](https://github.com/bozemanpass/example-todo-list), whose
+`stacks/todo/stack.yml` declares two wrapped containers built from the same repo:
+
+```yaml
+containers:
+  - name: bozemanpass/todo-frontend
+    wrapper: webapp
+    content-root: frontend
+  - name: bozemanpass/todo-backend
+    wrapper: node-service
+    content-root: backend
+```
+
+## 1. Build from your working tree
+
+Normally `stack` clones every repo a stack needs into the dev root
+(`STACK_REPO_BASE_DIR`, default `~/.config/stack/repos`) and builds from those clones. That is
+the wrong tree to develop in: `--git-pull` may move it under you, and it is not where your
+editor, branches, or IDE are pointed.
+
+Instead, identify the stack by **path** into your own checkout:
+
+```bash
+STACK=~/projects/example-todo-list/stacks/todo
+```
+
+When a stack is loaded from a git checkout that is not the dev-root clone of its repo,
+`Stack.repo_is_local_checkout()` reports true, and two things follow:
+
+- the stack's repo is never cloned or pulled — your tree is left exactly as you left it; and
+- every container whose source *is* that repo builds with **your working tree as the build
+  context**.
+
+The second point covers containers that name no `ref:` (both of the containers above), plus
+any container whose `ref:` resolves to the stack's own repo. A container that names some
+*other* repo still builds from the dev-root clone of that repo, as usual — the local-checkout
+rule applies to the tree you are developing in, not to the stack's dependencies.
+
+`content-root:` then narrows what is built: `frontend` for the front end image, `backend` for
+the API. Editing `frontend/src/App.tsx` changes only `bozemanpass/todo-frontend`.
+
+## 2. Build with an explicit build policy
+
+```bash
+stack prepare --stack $STACK --build-policy build
+```
+
+`--build-policy build` is not optional politeness here; the default `as-needed` policy will
+sometimes ignore your edits. To see why, recall how an image is identified
+([image-names.md](image-names.md)): the tag is the recipe repo's commit hash when the checkout
+is clean, and `stackdev-<hash of HEAD + the diff>` when it is dirty. Under `as-needed`,
+`prepare` reuses a matching local image or pulls a matching published one, and only builds if
+neither exists.
+
+That is exactly right for a clean tree and exactly wrong for a dirty one, because "dirty" is
+narrower than it sounds:
+
+- **Untracked files do not count.** The dirtiness check ignores untracked files, so a
+  brand-new component you have not `git add`ed leaves the tree "clean" — the expected tag is
+  the plain commit hash, and `as-needed` will happily pull the published image for that commit
+  from ghcr and deploy it in place of your work.
+- **Only unstaged changes feed the hash.** The `stackdev-` hash is computed from `git diff`,
+  i.e. tracked-but-unstaged modifications. Once you `git add` an edit it drops out of that
+  diff, so *every* staged-only state of a given commit produces the same `stackdev-` tag,
+  whatever the staged content is — and `as-needed` will reuse whichever image was built first
+  under that tag.
+
+`--build-policy build` skips the reuse-or-pull branch entirely and always builds.
+`--build-policy build-force` additionally builds without the container layer cache — reach for
+it when a build step caches something it should not have.
+
+Rebuild only what you touched:
+
+```bash
+stack prepare --stack $STACK --build-policy build \
+    --include-containers bozemanpass/todo-frontend
+```
+
+Either way the result is tagged `bozemanpass/todo-frontend:stack` locally, which is the name
+every deployment consumes.
+
+## 3. The local loop (compose)
+
+Create the deployment once:
+
+```bash
+stack init --stack $STACK --output spec.yml --deploy-to compose --map-ports-to-host localhost-same
+stack deploy --spec-file spec.yml --deployment-dir ~/deployments/todo
+stack manage --dir ~/deployments/todo start
+```
+
+Then, per edit:
+
+```bash
+stack prepare --stack $STACK --build-policy build --include-containers bozemanpass/todo-frontend
+stack manage --dir ~/deployments/todo stop
+stack manage --dir ~/deployments/todo start
+```
+
+### What `start` does with the image
+
+`deploy` rewrites each `image: <name>:stack` in the generated compose files to
+`<name>:<cluster-id>` (e.g. `bozemanpass/todo-frontend:stack-99544d5a11a0556e`) so that
+concurrent deployments on one host do not share a mutable tag. That deployment-private tag is
+(re)pointed at the current `:stack` image on every `start`, so a rebuild is picked up by a
+stop/start with no tag housekeeping on your part — you will see
+
+```
+Tagging bozemanpass/todo-frontend:stack to bozemanpass/todo-frontend:stack-99544d5a11a0556e...
+```
+
+in the `start` output whenever the image has actually changed, and nothing when it has not.
+
+Note that `manage reload` is a `compose restart`, which reuses the existing containers and so
+does *not* pick up a new image; use `stop` then `start`.
+
+## 4. The `k8s-kind` loop
+
+`--deploy-to k8s-kind` needs no registry: local images are copied into the kind cluster on
+every `up`, so the loop is just
+
+```bash
+stack prepare --stack $STACK --build-policy build
+stack manage --dir ~/deployments/todo-kind stop
+stack manage --dir ~/deployments/todo-kind start
+```
+
+with no tag surgery. Use it to check the Kubernetes *shape* of a deployment — pods, volumes,
+ingress — without a real cluster.
+
+## 5. The remote Kubernetes loop
+
+A remote cluster cannot see your local docker daemon, so the image has to travel through a
+registry. `--publish-images` is not the mechanism: it deliberately refuses `stackdev-`
+versions, because those images correspond to no commit and must never occupy a canonical,
+reproducible-looking tag.
+
+The mechanism is the deployment's **staging registry**, configured at `init` time:
+
+```bash
+stack init --stack $STACK --output k8s-spec.yml --deploy-to k8s \
+    --image-registry ghcr.io/bozemanpass \
+    --http-proxy-fqdn todo.example.com --http-proxy-target todo-list:3000
+stack deploy --spec-file k8s-spec.yml --deployment-dir ~/deployments/todo-k8s
+```
+
+`stack manage --dir ... push-images` tags whatever `:stack` currently points at as
+`<registry>/<name>:deploy-<last 8 of the deployment id>` and pushes it; manifest generation
+rewrites the pod images to exactly that reference. The tag is per-*deployment*, not
+per-*build*, so it does not care whether the image is a `stackdev-` build — which is what
+makes this the right path for uncommitted work.
+
+Per edit:
+
+```bash
+stack prepare --stack $STACK --build-policy build
+stack manage --dir ~/deployments/todo-k8s push-images
+stack manage --dir ~/deployments/todo-k8s stop
+stack manage --dir ~/deployments/todo-k8s start
+```
+
+Non-kind Kubernetes deployments are generated with `imagePullPolicy: Always`, so restarting
+re-pulls the (unchanged) `deploy-<id>` tag and gets the new content. No local tag needs
+removing, unlike the compose case.
+
+You need push access to the registry (`docker login`) and the cluster needs pull access —
+`stack` assumes credentials are configured on the cluster out of band, under the pull secret
+name `stack-image-registry`. See [image-names.md](image-names.md) for the full resolution
+order.
+
+## 6. Landing the change
+
+Everything above produces `stackdev-` images: unpublishable by construction, and correctly so
+— they cannot be reproduced from any commit. Once the work is committed and pushed, the tree
+is clean again, the expected tag becomes the recipe repo's commit hash, and the normal
+machinery takes over:
+
+```bash
+stack prepare --stack $STACK --build-policy build --publish-images --image-registry ghcr.io/bozemanpass
+```
+
+That image *is* reproducible from a commit, so a deployment elsewhere can find it with no
+staging registry at all — `prepare` on another machine computes the same tag and pulls it.
+If the build wrote or updated lock files (`stack.lock`), commit those too: they are what pins
+the remaining build inputs so that the recipe commit alone identifies the image content.
+
+## Summary
+
+| | Source of truth | Gets the new image by |
+| --- | --- | --- |
+| compose | local `:stack` tag | re-tagging `:<cluster-id>` at `start` |
+| `k8s-kind` | local `:stack` tag | image copy into kind on every `start` |
+| `k8s` | staging registry | `push-images`, then `imagePullPolicy: Always` on restart |
+
+## See Also
+
+- [image-names.md](image-names.md) — image naming and tagging in full, including `stackdev-`
+- [fetching-containers.md](fetching-containers.md) — build policies and prebuilt-image discovery
+- [stack-files.md](stack-files.md) — `stack.yml`, `content-root`, and lock files
+- [wrappers.md](wrappers.md) — how application source with no container build of its own is packaged
+- [from-laptop-to-production.md](from-laptop-to-production.md) — choosing a deployment target

docs/image-names.md

@@ -125,9 +125,10 @@ registry the cluster can reach (see step 5), and `init` warns accordingly.
 
 - **compose target:** each `image: <name>:stack` in the generated compose
   files is rewritten to `<name>:stack-<cluster-id>` so that concurrent
-  deployments on one host don't share a mutable tag. At `up` time the local
-  `<name>:stack` image is retagged to match (erroring with "did you run
-  stack prepare?" if absent).
+  deployments on one host don't share a mutable tag. At every `up` the local
+  `<name>:stack` image is retagged to match whenever the two names resolve to
+  different images, so a rebuild is picked up by a restart (erroring with "did
+  you run stack prepare?" when neither name exists locally).
 - **k8s targets:** pod files keep `<name>:stack`; translation happens at
   manifest-generation time (next step).
 

src/stack/deploy/compose/deploy_docker.py

@@ -18,14 +18,21 @@
 from pathlib import Path
 from python_on_whales import DockerClient, DockerException
 
-from stack.build.build_util import container_exists_locally
 from stack.deploy.deployer import Deployer, DeployerException, DeployerConfigGenerator
 from stack.deploy.deployment_context import DeploymentContext
 from stack.log import output_main, log_info
 from stack.opts import opts
 from stack.util import get_yaml, error_exit
 
 
+def _local_image_id(tag):
+    """The image ID a local tag resolves to, or None if the tag doesn't exist locally."""
+    try:
+        return DockerClient().image.inspect(tag).id
+    except Exception:
+        return None
+
+
 class DockerDeployer(Deployer):
     name: str = "compose"
     type: str
@@ -47,21 +54,38 @@ def __init__(
         self.compose_files = compose_files
         self.deployment_context = deployment_context
 
+    def _stage_local_images(self):
+        """Point this deployment's private image tags at the current locally built images.
+
+        `deploy` rewrites `<name>:stack` to `<name>:<cluster-id>` so that concurrent
+        deployments on one host don't share a mutable tag, so the tag has to be created
+        here.  Creating it only when absent is not enough: a rebuild (`stack prepare`)
+        moves `:stack` to a new image while the deployment tag still names the old one,
+        which would silently keep running the previous build.  So re-tag whenever the two
+        names disagree, which is also the edit-build-restart loop for a compose deployment
+        (see docs/developing-applications.md).
+        """
+        for compose_file in self.compose_files:
+            parsed_file = get_yaml().load(open(compose_file, "r"))
+            for svc_name in parsed_file.get("services", {}):
+                image = parsed_file["services"][svc_name].get("image")
+                if not image or not image.endswith(self.deployment_context.id):
+                    continue
+                stack_image = image.replace(f":{self.deployment_context.id}", ":stack")
+                stack_image_id = _local_image_id(stack_image)
+                if stack_image_id is None:
+                    # No locally built image: the staged tag is all there is to run.
+                    if _local_image_id(image) is not None:
+                        continue
+                    error_exit(f"Cannot find {image} or {stack_image} locally. Did you run 'stack prepare'?")
+                if stack_image_id == _local_image_id(image):
+                    continue
+                log_info(f"Tagging {stack_image} to {image}...")
+                self.docker.tag(stack_image, image)
+
     def up(self, detach, skip_cluster_management, services):
         if not opts.o.dry_run:
-            for compose_file in self.compose_files:
-                parsed_file = get_yaml().load(open(compose_file, "r"))
-                if "services" in parsed_file:
-                    for svc_name in parsed_file["services"]:
-                        image = parsed_file["services"][svc_name].get("image")
-                        if image and image.endswith(self.deployment_context.id):
-                            if not container_exists_locally(image):
-                                stack_image = image.replace(f":{self.deployment_context.id}", ":stack")
-                                if container_exists_locally(stack_image):
-                                    log_info(f"Tagging {stack_image} to {image}...")
-                                    self.docker.tag(stack_image, image)
-                                else:
-                                    error_exit(f"Cannot find {image} or {stack_image} locally. Did you run 'stack prepare'?")
+            self._stage_local_images()
             try:
                 return self.docker.compose.up(detach=detach, services=services)
             except DockerException as e: