chore(docker): add a .dockerignore to keep the build context small - #164
Merged
Conversation
Every worker builds from the repo root, so each `docker build` sent the entire working tree to the daemon before running any instruction. On a checkout with local test venvs and downloaded model checkpoints that is ~11 GB per build, across 107 Dockerfiles. Excludes .git, .github, per-worker local_tests/ and .venv/ scratch, and compiled bytecode. Nothing listed is referenced by any COPY/ADD -- checked by grepping every Dockerfile: worker COPY lines name specific files rather than directories, and the two directories copied wholesale (annotation_utilities, worker_client) only lose stale __pycache__. Verified in practice: the eight ML worker images in #163 were built with this file in place, and each one loaded its baked-in models and ran a real job against a live NimbusImage stack. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DGVianXJET5VVja76jahvj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every worker in this repo builds from the repo root (
docker build . -f workers/annotations/<name>/Dockerfile), and Docker sends the entire build context to the daemon before running a single instruction. There was no.dockerignore, so that context is the whole working tree.On a real development checkout that is ~11 GB, essentially all of it from files no image ever uses:
workers/annotations/sam_automatic_mask_generator/local_tests.venvwith torch/CUDA + a 2.6 GBsam_vit_h_4b8939.pthcheckpointworkers/annotations/sam2_fewshot_segmentation/local_tests.gitThat cost is paid on every build, by all 107 Dockerfiles in the repo.
What this excludes
.git,.github, per-workerlocal_tests/and.venv/scratch, and compiled bytecode (__pycache__,*.py[cod]).Why it is safe
Nothing listed is referenced by any
COPY/ADD— checked by grepping every Dockerfile in the repo:COPY/ADDanywhere mentionslocal_tests,.venv,__pycache__or.git.COPYlines name specific files (entrypoint.py,environment.yml,utils.py, …), not whole worker directories, so excluding a subdirectory of a worker cannot remove anything a build needs.annotation_utilitiesandworker_client— only lose stale__pycache__, which is regenerated inside the image anyway.tests/is deliberately not excluded: the*_testimages copy it.Verified in practice as well as by inspection: the eight ML worker images in #163 were all built with this file in place, and each one loaded its baked-in model cache and ran a real job against a live NimbusImage stack.
Note
This is orthogonal to #163 (image size) — this is build context transfer, which affects every build in the repo including CPU workers. Split out deliberately so #163 stays focused.
Generated by Claude Code