Skip to content

--score-push reports success but the score is never published (0.4.51; 0.4.22 works) #445

Description

@stephrobert

Describe the bug

plumber analyze --score-push reports ✓ Plumber Score published, exits 0, and
the score never appears: the badge stays Plumber Score: unknown and the page
says No score published yet for this project.

This is reproducible on three different repositories, all pushed with
plumber 0.4.51. A fourth repository of the same owner, whose workflow still
pins 0.4.22, displays its score normally — which is why I suspect a
regression between the two rather than an account or permission problem.

github.com/stephrobert/collection-scaleway   0.4.51   unknown
github.com/stephrobert/collection-exoscale   0.4.51   unknown
github.com/stephrobert/collection-outscale   0.4.51   unknown
github.com/stephrobert/feint                 0.4.22   A

All four are public repositories owned by the same GitHub account, pushed from
GitHub Actions with id-token: write.

To Reproduce

  1. Public GitHub repository, workflow triggered on push to the default branch.
  2. Job with permissions: { contents: read, id-token: write }.
  3. Install the release binary for v0.4.51 (checksum verified).
  4. Run plumber analyze --min-score B --score-push.
  5. The step succeeds and prints the "published" line.
  6. Fetch the badge or the page:
curl -s https://score.getplumber.io/github.com/stephrobert/collection-scaleway.svg \
  | grep -o 'aria-label="[^"]*"'
# aria-label="Plumber Score: unknown"

curl -s https://score.getplumber.io/github.com/stephrobert/collection-scaleway \
  | grep -o 'No score published yet[^<]*'
# No score published yet for this project. Run

Checked again more than three hours after the push, so this is not propagation
delay.

Expected behavior

Either the badge shows the score that was pushed, or the command reports
that the push did not succeed. What makes this costly is not the missing badge:
it is that the CLI reports success. A caller has no way to tell "published" from
"accepted and dropped", so a green pipeline says the badge works when it does
not.

Even a non-zero exit or a warning naming what the backend answered would be
enough to act on.

Relevant versions and configuration files

  • OS: ubuntu-24.04 GitHub-hosted runner
  • Not GitLab — GitHub.com, public repositories, GitHub Actions
  • Plumber version: 0.4.51 (failing), 0.4.22 (working, other repository)
  • Installed from the release asset plumber-linux-amd64, SHA-256 verified
    against the published checksums.txt

.plumber.yaml (excerpt — the full file is public in the repository):

rules:
  pipelineComponentsMustBePinned:
    enabled: true
  trustedGithubActions:
    - step-security/harden-runner
    - jdx/mise-action
    - ossf/scorecard-action
    - google/osv-scanner-action
    - anchore/sbom-action
    - stephrobert/setup-feint
    - sigstore/cosign-installer
    - actions/attest-build-provenance
    - docker/login-action
    - docker/build-push-action
    - google/clusterfuzzlite
  branchMustBeProtected:
    enabled: false

Workflow step:

permissions:
  contents: read
  id-token: write
steps:
  - name: Plumber
    run: plumber analyze --min-score B --score-push

Relevant logs

From the run on 92f5a177:

plumber version 0.4.51
  ✓ Third-party actions must be pinned by commit SHA
    • GitHub API metadata enrichment unavailable (authentication token not found
      for host github.com), so the known-CVE, archived-repo, ref-existence and
      star-count checks were skipped for every action
  Status: PASSED ✓ (score A — 100.0/100 pts, required ≥ B)
 Plumber Score
✓ Plumber Score published: https://score.getplumber.io/github.com/stephrobert/collection-scaleway

The runner's egress monitor confirms the request left the machine and reached
the service:

domain resolved: score.getplumber.io., ip address: 216.150.16.129, TTL: 300
endpoint called ip address:port 216.150.16.129:443, domain: score.getplumber.io.,
  pid: 2650, process: plumber

So the call is made and answered; the score is simply not visible afterwards.

I have not run with --verbose on the failing version yet. If a verbose run
would help, tell me what to look for and I will attach it — the workflow is
public and I can re-run it on demand.

Additional context

Reading the release notes, the push path changed between the two versions I can
compare: v0.4.37 added a --platform feature that pushes results per policy,
v0.4.38 reworked it, and v0.4.50 / v0.4.51 changed the pushed body
(push effective_config as the flat per-provider controls map, assert display metadata on the decoded push body). That is a guess from the changelog, not a
diagnosis — I have no visibility into what the backend received.

Public evidence, if useful:

  • the workflow: https://github.com/stephrobert/collection-scaleway/blob/main/.github/workflows/plumber.yml
  • the run quoted above:
    https://github.com/stephrobert/collection-scaleway/actions/runs/34052132127

Happy to test a patched build or a flag against these four repositories, which
give a working and a non-working case side by side.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions