Skip to content

[BUG] s5cmd action intermittently fails: peak/action-setup-s5cmd hits unauthenticated GitHub API rate limit #85

Description

@lowlydba

What's broken

The s5cmd composite action's Install s5cmd step (peak/action-setup-s5cmd@bde462e) intermittently fails:

jq: error (at <stdin>:1): Cannot index string with string "tag_name"
##[error]Process completed with exit code 5.

Observed here: https://github.com/OvertureMaps/stac/actions/runs/32257838838/job/96084476819

Root cause

peak/action-setup-s5cmd's find_release_asset step resolves the download URL with an unauthenticated call:

curl -s https://api.github.com/repos/peak/s5cmd/releases | jq --arg tag "${tag}" ... '.[] | select(.tag_name==$tag) | ...'

GitHub rate-limits unauthenticated api.github.com calls to 60 requests/hour per source IP, and GitHub-hosted runners share IP pools across the whole fleet, so that quota gets exhausted by unrelated jobs. When rate-limited, the API returns {"message": "API rate limit exceeded..."} (an object, not the expected array), and jq '.[]' chokes trying to index its string values, exiting 5.

Confirmed the API itself is healthy right now (a fresh unauthenticated call just returned 200 with X-RateLimit-Remaining: 59), so this is intermittent, not a permanent break: it fails whenever the runner's shared IP has already burned its hourly quota.

Fix

Since this lives in peak/action-setup-s5cmd itself rather than anything OvertureMaps controls, options are to either pass a GITHUB_TOKEN to that step's curl call (raises the quota to 5,000/hour, authenticated) if the action supports it, or wrap the s5cmd composite action's call to it with a retry/backoff so a transient 60/hour exhaustion doesn't fail the whole publish job.

Metadata

Metadata

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions