diff --git a/.github/workflows/publish-release.yml b/.github/workflows/publish-release.yml index 031bd405f..e8f0a3413 100644 --- a/.github/workflows/publish-release.yml +++ b/.github/workflows/publish-release.yml @@ -206,13 +206,27 @@ jobs: # Pushed by digest and deliberately untagged. A tag written here would name one architecture, # and the two jobs for one image would race to own it, so the last to finish would decide what # the tag meant. + # + # zstd, not gzip. A pull is already a compressed transfer, so this is not about compressing + # something that was not; it is a better algorithm for the same job. Measured on + # `agent-computer`, the image that matters: 962 MB becomes 886 MB, and it inflates several + # times faster, which is worth more than the 8% on 2 GB of Chromium. `force-compression` + # is what reaches the layers that came from somebody else's registry, which is where almost + # all of the bytes are; without it only our own thin layers change and the saving rounds to + # nothing. The cost is that these images no longer share layers with a gzip pull of the same + # base, and that a client which cannot read zstd cannot read them, which is why this is here + # and not on the `openbot` image above: that one is pulled by servers with whatever they have, + # and these are pulled by an installer that ships Podman. - id: push uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 with: context: . file: ${{ matrix.image }}/Dockerfile platforms: linux/${{ matrix.platform.arch }} - outputs: type=image,name=ghcr.io/copilotkit/openbot-${{ matrix.image }},push-by-digest=true,name-canonical=true,push=true + # `oci-mediatypes=true` because zstd layers have no Docker-schema2 media type to be + # described by. It is buildx's default for this exporter and is stated rather than + # assumed, since the push fails without it and the reason would not be obvious. + outputs: type=image,name=ghcr.io/copilotkit/openbot-${{ matrix.image }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true,compression=zstd,force-compression=true # Scoped per image and per architecture. One shared scope would have ten builds # overwriting each other's cache and none of them reading their own. cache-from: type=gha,scope=${{ matrix.image }}-${{ matrix.platform.arch }} diff --git a/CHANGELOG.md b/CHANGELOG.md index c68326711..f56092d13 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,19 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged. ## Unreleased +### The published service images are zstd rather than gzip + +An image pull was already a compressed transfer, so this is not compression where there was none; it +is a better algorithm for the same job. `agent-computer` goes from 962 MB to 886 MB on the wire, and +zstd inflates several times faster, which on 2 GB of Chromium is worth more than the 8%. The saving +comes from recompressing the layers that arrived from somebody else's registry, where nearly all of +the bytes are, so the images no longer share layers with a gzip pull of the same base. + +This applies to the five `ghcr.io/copilotkit/openbot-` images and not to +`ghcr.io/copilotkit/openbot`. Reading a zstd layer needs a client that supports it, which Podman and +current containerd do; the single image is pulled by deployments running whatever they have, so it +stays gzip. + ### A release publishes every service's image, not just the one `ghcr.io/copilotkit/openbot` was the only image a release produced, so anything running the Compose