ci: scope Depot usage — per-release benchmarks + free runners for lint jobs - #47
Open
dhogaivannan wants to merge 6 commits into
Open
ci: scope Depot usage — per-release benchmarks + free runners for lint jobs#47dhogaivannan wants to merge 6 commits into
dhogaivannan wants to merge 6 commits into
Conversation
Trimmed port of upstream's benchmarks workflow for validating Monad releases against the previous one. Runs automatically on v*.*.*-monad-v*.*.* tag pushes and manually via workflow_dispatch — no schedule, so zero standing Depot cost (~1h of 32-core per release). Three bench repos, forge test/fuzz/build benchmarks only, and no publish/commit job — results go to the run log and an artifact. The previous release to compare against is resolved dynamically from git tags (same logic as release.yml's PREV_STABLE, fallback v1.7.1-monad-v1.0.0) and installed via foundryup --network monad; 'local' is built from the workflow ref with the same dist profile and feature set as release binaries. Verified against the live v1.7.1-monad-v1.0.0 release: asset naming matches what foundryup downloads, and the monad install path needs no extra tooling. Part of DO-322 (scope Depot usage to Monad releases).
These 8 jobs ran on depot-ubuntu-latest (2 vCPU, billed). GitHub-hosted ubuntu-latest is free for public repos and gives 4 vCPU, so this is both cheaper and faster. The heavy test matrix in test.yml stays on Depot 16-core runners. Part of DO-322 (scope Depot usage to Monad releases).
dhogaivannan
force-pushed
the
ci/monad-release-benchmarks
branch
from
July 11, 2026 23:45
8d8e8a0 to
b800a3d
Compare
clippy::redundant_else (deny-warnings) — the if branch ends in panic!, so the else block is unnecessary. Surfaced by the first clippy run that actually executed after the runner move; same result on any runner.
Run pull-request test shards on GitHub-hosted Linux while Depot is disconnected. Drop the obsolete test-utils workaround after syncing the base and narrowly allow Chisel’s existing async stack-frame lint without changing runtime behavior.
Run the Linux all-tests shard with the complete feature set so Monad-specific tests deserialize Monad networks and hardforks. Keep the shared Forge fixture on its pinned forge-std revision instead of reinstalling the dependency during setup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two CI changes for Linear DO-322 (scope Depot usage to Monad releases), one commit each:
1. Per-release benchmarks workflow (
benchmarks.yml, new)Trimmed port of upstream's benchmarks workflow for validating releases against the previous one:
v*.*.*-monad-v*.*.*tag push (automatic, once per release) +workflow_dispatch— no schedule, zero standing Depot cost. One run ≈ 1 hour on a 32-core Depot runner.release.ymlwith no ordering dependency:localis built from the tagged commit, and the baseline is the previous release's published binaries.release.yml'sPREV_STABLE; fallbackv1.7.1-monad-v1.0.0) — nothing to bump after each release.--nmcfilters carried over from upstream to skip tests known to be flaky across versions.forge_test,forge_fuzz_test,forge_build_no_cache,forge_build_with_cache. No isolate/coverage/symbolic.How versions work: the monad-branch
foundry-benchpredates upstream's built-inlocalversion, so the workflow handles versions itself — release tags are pre-installed viafoundryup --install <tag> --network monad(which lands in$FOUNDRY_DIR/versions/<tag>, exactly wherefoundryup --uselooks), andlocalis built with the samedistprofile + feature set asrelease.yml(incl. jemalloc on x86_64 Linux) and registered asversions/local.foundry-benchruns without--force-install, so it only switches between pre-installed versions.Ad-hoc usage:
2. Lint/check jobs → free GitHub-hosted runners (
ci.yml)Moves the 8 lint/check jobs (doctest, no-default-features, typos, shellcheck, clippy, rustfmt, forge-fmt, crate-checks) from
depot-ubuntu-latestto plainubuntu-latest:depot-ubuntu-latestis the 2-vCPU Depot base runner, billed per minute; GitHub-hostedubuntu-latestis free for public repos and provides 4 vCPU — cheaper and faster.test.ymlmatrix (nextest, 16-core) intentionally stays on Depot.Verification
yaml.safe_loadparses both files;actionlintclean onci.yml; its only note onbenchmarks.ymlis thedepot-ubuntu-24.04-32label it can't know about (expected for third-party runners).foundry-benchsupports--reposorg/repo[:rev][ args]syntax and all four benchmark names;foundryupmonad installs extract toversions/<tag>;use()copiesforge,cast,anvil,chisel— all four are built forlocal.v1.7.1-monad-v1.0.0release: asset naming matches the URLfoundryup --network monadconstructs (foundry_v1.7.1-monad-v1.0.0_linux_amd64.tar.gzexists), and the monad install path does a plain download + tar check (no attestation/sha step needing extra tooling). Old-scheme releases (v1.5.0-monad.0.x.0) can never be selected — the tag glob doesn't match their naming.test.ymlmatrix checks still need the repo re-added to the Depot app before they can pass.🤖 Written with Claude Code