Skip to content

ci: scope Depot usage — per-release benchmarks + free runners for lint jobs - #47

Open
dhogaivannan wants to merge 6 commits into
monadfrom
ci/monad-release-benchmarks
Open

ci: scope Depot usage — per-release benchmarks + free runners for lint jobs#47
dhogaivannan wants to merge 6 commits into
monadfrom
ci/monad-release-benchmarks

Conversation

@dhogaivannan

@dhogaivannan dhogaivannan commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Two CI changes for Linear DO-322 (scope Depot usage to Monad releases), one commit each:

1. Per-release benchmarks workflow (benchmarks.yml, new)

Trimmed port of upstream's benchmarks workflow for validating releases against the previous one:

  • Triggers: v*.*.*-monad-v*.*.* tag push (automatic, once per release) + workflow_dispatch — no schedule, zero standing Depot cost. One run ≈ 1 hour on a 32-core Depot runner.
  • On a tag push it runs in parallel with release.yml with no ordering dependency: local is built from the tagged commit, and the baseline is the previous release's published binaries.
  • The previous release is resolved dynamically from git tags (same logic as release.yml's PREV_STABLE; fallback v1.7.1-monad-v1.0.0) — nothing to bump after each release.
  • 3 bench repos (solady, ithacaxyz/account, uniswap/v4-core); --nmc filters carried over from upstream to skip tests known to be flaky across versions.
  • 4 benchmarks: forge_test, forge_fuzz_test, forge_build_no_cache, forge_build_with_cache. No isolate/coverage/symbolic.
  • No publish/commit job — results go to the run log and a run artifact; nothing is written back to the repo.

How versions work: the monad-branch foundry-bench predates upstream's built-in local version, so the workflow handles versions itself — release tags are pre-installed via foundryup --install <tag> --network monad (which lands in $FOUNDRY_DIR/versions/<tag>, exactly where foundryup --use looks), and local is built with the same dist profile + feature set as release.yml (incl. jemalloc on x86_64 Linux) and registered as versions/local. foundry-bench runs without --force-install, so it only switches between pre-installed versions.

Ad-hoc usage:

gh workflow run benchmarks.yml --ref monad
# or explicitly: -f versions=v1.7.1-monad-v1.0.0,local

2. Lint/check jobs → free GitHub-hosted runners (ci.yml)

Moves the 8 lint/check jobs (doctest, no-default-features, typos, shellcheck, clippy, rustfmt, forge-fmt, crate-checks) from depot-ubuntu-latest to plain ubuntu-latest:

  • depot-ubuntu-latest is the 2-vCPU Depot base runner, billed per minute; GitHub-hosted ubuntu-latest is free for public repos and provides 4 vCPU — cheaper and faster.
  • The heavy test.yml matrix (nextest, 16-core) intentionally stays on Depot.
  • Side benefit: these checks run immediately even while the repo is disconnected from the Depot app, so PRs get partial CI signal today. This PR's own run is the live test — if a compile-heavy job brushes its 30-minute timeout on 4 vCPU, the fallback is bumping that job's timeout or pinning it back to a Depot label.

Verification

  • yaml.safe_load parses both files; actionlint clean on ci.yml; its only note on benchmarks.yml is the depot-ubuntu-24.04-32 label it can't know about (expected for third-party runners).
  • Confirmed against branch code: foundry-bench supports --repos org/repo[:rev][ args] syntax and all four benchmark names; foundryup monad installs extract to versions/<tag>; use() copies forge,cast,anvil,chisel — all four are built for local.
  • Confirmed against the live v1.7.1-monad-v1.0.0 release: asset naming matches the URL foundryup --network monad constructs (foundry_v1.7.1-monad-v1.0.0_linux_amd64.tar.gz exists), and the monad install path does a plain download + tar check (no attestation/sha step needing extra tooling). Old-scheme releases (v1.5.0-monad.0.x.0) can never be selected — the tag glob doesn't match their naming.
  • Note: the test.yml matrix checks still need the repo re-added to the Depot app before they can pass.

🤖 Written with Claude Code

@dhogaivannan dhogaivannan changed the title ci: add manual per-release benchmarks workflow ci: scope Depot usage — per-release benchmarks + free runners for lint jobs Jul 11, 2026
Trimmed port of upstream's benchmarks workflow for validating Monad
releases against the previous one. Runs automatically on
v*.*.*-monad-v*.*.* tag pushes and manually via workflow_dispatch —
no schedule, so zero standing Depot cost (~1h of 32-core per release).

Three bench repos, forge test/fuzz/build benchmarks only, and no
publish/commit job — results go to the run log and an artifact.

The previous release to compare against is resolved dynamically from git
tags (same logic as release.yml's PREV_STABLE, fallback
v1.7.1-monad-v1.0.0) and installed via foundryup --network monad;
'local' is built from the workflow ref with the same dist profile and
feature set as release binaries. Verified against the live
v1.7.1-monad-v1.0.0 release: asset naming matches what foundryup
downloads, and the monad install path needs no extra tooling.

Part of DO-322 (scope Depot usage to Monad releases).
These 8 jobs ran on depot-ubuntu-latest (2 vCPU, billed). GitHub-hosted
ubuntu-latest is free for public repos and gives 4 vCPU, so this is both
cheaper and faster. The heavy test matrix in test.yml stays on Depot
16-core runners.

Part of DO-322 (scope Depot usage to Monad releases).
@dhogaivannan
dhogaivannan force-pushed the ci/monad-release-benchmarks branch from 8d8e8a0 to b800a3d Compare July 11, 2026 23:45
dhogaivannan and others added 4 commits July 13, 2026 13:44
clippy::redundant_else (deny-warnings) — the if branch ends in panic!,
so the else block is unnecessary. Surfaced by the first clippy run that
actually executed after the runner move; same result on any runner.
Run pull-request test shards on GitHub-hosted Linux while Depot is disconnected. Drop the obsolete test-utils workaround after syncing the base and narrowly allow Chisel’s existing async stack-frame lint without changing runtime behavior.
Run the Linux all-tests shard with the complete feature set so Monad-specific tests deserialize Monad networks and hardforks. Keep the shared Forge fixture on its pinned forge-std revision instead of reinstalling the dependency during setup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants