The downstream smoke check (scripts/test-downstream.sh, the gate from step 4 of RELEASING.md) returned 1 failed / 4 skipped / 0 passed on a clean run for the v1.12.0 release, even though the actual SDK changes were purely additive. Two issues conflate:
1. pip install <repo>[dev] fails silently when the repo doesn't declare a dev extra
Line 155 of scripts/test-downstream.sh runs "$PIP" install --quiet "$repo_path[dev]" and then continues to the test step. pip prints WARNING: <repo> 0.x.0 does not provide the extra 'dev', exits 0, and leaves pytest uninstalled. The next step ("$venv_dir/bin/pytest" --ignore=tests/integration ...) then errors with No such file or directory and the repo is reported as failed.
Concretely, crewai-colony's pyproject.toml only declares [project.optional-dependencies] async, not dev. So the smoke check fails on every release regardless of whether the SDK changed anything.
Two ways to fix:
- Add an explicit dev-deps install in the script (e.g.
pip install pytest pytest-asyncio after the [dev] install, or unconditionally), so the test runner is guaranteed to be present even if the downstream's [dev] extra has been renamed or removed.
- OR detect the missing
dev extra and skip the repo cleanly with a clear message, the same way unfound repos are skipped, so it doesn't read as a release-blocking failure.
2. Repos cloned under .<repo>/ (dot-prefix workspace convention) aren't discovered
The other four downstream repos — langchain-colony, openai-agents-colony, smolagents-colony, pydantic-ai-colony — are checked out as .langchain-colony/, .openai-agents-colony/, etc. relative to the workspace root. The script's discovery list (lines 30-55 area) only checks:
$COLONY_DOWNSTREAM_DIR/<repo>/../<repo>/ (sibling)/tmp/<repo>/
None of these match the actual layout, so all four are reported as "not found locally" and skipped silently. The release thus gets zero downstream signal — which is what the script exists to provide.
Fix: add ../.<repo>/ to the discovery list. That keeps the existing layouts working and picks up the dot-prefix convention without requiring environment-variable setup.
Why this matters now
For the v1.12.0 vault release the actual risk was low (pure-additive new methods, no signature changes), but the script's report — Do not release — is misleading on a healthy release and reassuring on an unhealthy one. The check needs to be either fixed or de-blockified before the next class-of-bug-it-catches (return-type changes, like the v1.7.0 → v1.7.1 fiasco that motivated it in the first place) actually ships.
cc @arch-colony for visibility — the equivalent JS check (@thecolony/sdk) doesn't have this gap, but the Python release flow needs a fix.
The downstream smoke check (
scripts/test-downstream.sh, the gate from step 4 of RELEASING.md) returned 1 failed / 4 skipped / 0 passed on a clean run for the v1.12.0 release, even though the actual SDK changes were purely additive. Two issues conflate:1.
pip install <repo>[dev]fails silently when the repo doesn't declare adevextraLine 155 of
scripts/test-downstream.shruns"$PIP" install --quiet "$repo_path[dev]"and then continues to the test step. pip printsWARNING: <repo> 0.x.0 does not provide the extra 'dev', exits 0, and leaves pytest uninstalled. The next step ("$venv_dir/bin/pytest" --ignore=tests/integration ...) then errors withNo such file or directoryand the repo is reported as failed.Concretely,
crewai-colony'spyproject.tomlonly declares[project.optional-dependencies]async, notdev. So the smoke check fails on every release regardless of whether the SDK changed anything.Two ways to fix:
pip install pytest pytest-asyncioafter the[dev]install, or unconditionally), so the test runner is guaranteed to be present even if the downstream's[dev]extra has been renamed or removed.devextra and skip the repo cleanly with a clear message, the same way unfound repos are skipped, so it doesn't read as a release-blocking failure.2. Repos cloned under
.<repo>/(dot-prefix workspace convention) aren't discoveredThe other four downstream repos —
langchain-colony,openai-agents-colony,smolagents-colony,pydantic-ai-colony— are checked out as.langchain-colony/,.openai-agents-colony/, etc. relative to the workspace root. The script's discovery list (lines 30-55 area) only checks:$COLONY_DOWNSTREAM_DIR/<repo>/../<repo>/(sibling)/tmp/<repo>/None of these match the actual layout, so all four are reported as "not found locally" and skipped silently. The release thus gets zero downstream signal — which is what the script exists to provide.
Fix: add
../.<repo>/to the discovery list. That keeps the existing layouts working and picks up the dot-prefix convention without requiring environment-variable setup.Why this matters now
For the v1.12.0 vault release the actual risk was low (pure-additive new methods, no signature changes), but the script's report —
Do not release— is misleading on a healthy release and reassuring on an unhealthy one. The check needs to be either fixed or de-blockified before the next class-of-bug-it-catches (return-type changes, like the v1.7.0 → v1.7.1 fiasco that motivated it in the first place) actually ships.cc @arch-colony for visibility — the equivalent JS check (
@thecolony/sdk) doesn't have this gap, but the Python release flow needs a fix.