publish.yml: dry_run defaults to true and a skipped publish still reports job success

[khaliqgant]

Problem

publish.yml defaults dry_run to true. Dispatching without explicitly passing dry_run=false runs a dry run, skips the Publish step, and still reports the job as success.

The failure is silent in the worst direction: the workflow, the job, and the run all show green, and nothing is published.

How it presented

Dispatched during a live production outage:

gh workflow run publish.yml -f version=0.1.6 -f tag=latest

Result:

run 32667097852 -> completed/success
job "Publish to npm": success

Registry afterwards: latest = 0.1.5, and npm view @agent-relay/sandbox@0.1.6 did not resolve.

Only inspecting individual step conclusions revealed what actually happened:

step Check release tag = skipped
step Publish           = skipped
step Tag release       = skipped

Every meaningful step was gated on if: ${{ github.event.inputs.dry_run != 'true' }}, and the input's declared default is default: true.

I initially assumed registry propagation lag, because a previous publish had genuinely taken ~2.5 minutes to appear. That is a plausible and wrong explanation available for exactly as long as it takes to waste time on it. During an outage that cost about six minutes before I checked step-level conclusions.

Why the default itself is defensible but the reporting is not

Defaulting to a dry run is a reasonable fail-safe: an accidental dispatch should not publish. The defect is that a dry run is indistinguishable from a real publish at every level a caller normally checks — run conclusion, job conclusion, and the workflow's name in the UI all say the same thing either way.

This is the same class of bug the codebase has been fixing all day elsewhere: a success-shaped signal for work that did not happen. gh run view --json status,conclusion is the ordinary way to check a dispatch, and it cannot distinguish these two outcomes.

Suggested fixes, cheapest first

1. Make the job name or summary state the mode. A step that writes DRY RUN — nothing was published / PUBLISHED <version> to <tag> into $GITHUB_STEP_SUMMARY makes the outcome legible without changing behaviour.
2. Fail the run if a real publish was intended but skipped. A final assertion step that, when dry_run != 'true', verifies the version actually resolves on the registry — and exits non-zero otherwise — turns this into a loud failure. That also catches genuine registry-side publish failures, which today would be invisible for the same reason.
3. Consider flipping the default to false with a required explicit confirmation, or renaming the input so the safe default is obvious at the dispatch site. This is a judgement call and the current default may be the right one; items 1 and 2 fix the reporting regardless of which way it goes.

Related

#29 established package.json as the sole version authority and made the workflow take an exact version. That work was correct and is not what caused this. This is specifically about the dry-run gate being invisible in the run's reported outcome.

The real publish for 0.1.6 was subsequently dispatched with dry_run=false as run 32667419706.