Skip to content

feat: zero-config github-token default and branded comments via the relay - #9

Merged
soroushm merged 8 commits into
mainfrom
chore/vendor-bench-at-release
Jul 23, 2026
Merged

feat: zero-config github-token default and branded comments via the relay#9
soroushm merged 8 commits into
mainfrom
chore/vendor-bench-at-release

Conversation

@soroushm

@soroushm soroushm commented Jul 22, 2026

Copy link
Copy Markdown
Member

Closes #8.

The results comment now works with zero consumer configuration, and can carry the bench brand when the public bench GitHub App is installed.

Changes

  • github-token defaults to ${{ github.token }} — a consumer's whole integration is the uses: block plus the gate inputs; the sticky comment posts with the workflow's automatic runtime token. Explicit '' still opts out; passing an App token still overrides the author.
  • New branded input (default 'true'): when the job grants id-token: write and the bench GitHub App is installed on the repo, the comment is posted as the bench bot through the OIDC-verified relay at api.bench.soroush.tech (src/relay.ts; relay worker lives in [RFC] Branded benchmark PR comments via a GitHub App and a hosted comment relay core#308). Any relay failure — app not installed, no OIDC permission, relay down — logs the reason and falls back to the direct github-token comment. The gate verdict is never affected by comment delivery.
  • README: tokenless usage example, branded-comment docs, inputs table.
  • Version → 1.1.0 + release-notes/1.1.0.md (default flip = contract change, minor bump).

Verification

  • 37 tests at 100% coverage (relay success/4xx/OIDC-rejection/5xx/no-permission paths covered), typecheck and build green.
  • Fallback ordering in the entrypoint: relay attempt only on PRs with branded not 'false'; every failure path degrades to the existing direct-comment code, which is unchanged and already tested.
  • After merge: dispatch the Release workflow for 1.1.0, then the Marketplace listing can be done on that release.

Summary by CodeRabbit

  • New Features

    • PR result comments can now be authored by the bench bot through a verified relay when configured.
    • Added the branded option to control branded commenting.
    • Comments fall back to direct posting if relay services are unavailable.
  • Changed Defaults

    • PR comments work by default using the workflow token.
    • To disable comments, explicitly set an empty github-token.
  • Documentation

    • Updated setup guidance for permissions, fork behavior, relay commenting, and fallback handling.
    • Added release notes for version 1.1.0.

…elay - close #8

- github-token defaults to the workflow runtime token, so the PR comment
  works with no setup; explicit '' still skips commenting
- new branded input (default true): post the comment as the bench bot via
  the OIDC-verified relay at api.bench.soroush.tech (soroush-tech/core#308);
  any relay failure logs the reason and falls back to the direct comment,
  never affecting the gate verdict
- covered relay client (src/relay.ts) with injected deps; version 1.1.0 with
  release notes
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@soroushm, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2a2ab85a-591f-4802-afb3-15006c36b610

📥 Commits

Reviewing files that changed from the base of the PR and between 95cabce and 728c6a9.

📒 Files selected for processing (9)
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • README.md
  • action.yml
  • package.json
  • release-notes/1.1.0.md
  • src/index.ts
  • src/relay.test.ts
  • src/relay.ts
📝 Walkthrough

Walkthrough

The action adds OIDC-authenticated branded PR commenting through a hosted relay, with direct-comment fallback and new input defaults. CI and release workflows adjust dependency installation, vendoring validation, and release SHA propagation. Version 1.1.0 documentation and release notes describe the updated behavior.

Changes

Branded PR commenting

Layer / File(s) Summary
Relay client and response handling
src/relay.ts, src/relay.test.ts
Adds OIDC token acquisition, authenticated report submission, explicit HTTP error handling, and Vitest coverage.
Action inputs and comment integration
action.yml, src/index.ts, README.md, release-notes/1.1.0.md, package.json
Adds branded posting and runtime-token defaults, integrates relay posting with direct fallback, and documents the 1.1.0 behavior.
Release artifact and workflow handling
.github/workflows/ci.yml, .github/workflows/release.yml, scripts/release-commit.mjs
Disables lifecycle scripts during dependency installation, simplifies vendor validation, and passes validated release SHAs through workflow outputs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Action as bench-action
  participant OIDC as GitHub OIDC
  participant Relay as Hosted relay
  participant GitHub as GitHub API

  Action->>OIDC: Request token with relay audience
  OIDC-->>Action: Return OIDC token
  Action->>Relay: Post repository, PR number, and report
  Relay-->>Action: Return success or failure
  Action->>GitHub: Directly upsert comment when relay fails
Loading

Possibly related issues

  • soroush-tech/core issue 308: Covers the hosted relay, branded PR comments, OIDC authentication, and fallback behavior implemented here.
  • soroush-tech/core issue 309: Covers the relay client and hosted endpoint interaction added by this change.
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The token-default and docs updates match #8, but the PR also changes the entrypoint for branded relay comments, which #8 said not to do. Split the relay/comment-authoring work into a separate PR, or remove it here so this PR only covers the github-token default, docs, and version bump.
Out of Scope Changes check ⚠️ Warning Relay implementation, CI/release workflow edits, and entrypoint changes go beyond #8's token-default/docs/version-bump scope. Remove unrelated relay/workflow changes from this PR or link a broader issue that explicitly covers them.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: defaulting github-token and adding branded relay comments.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/vendor-bench-at-release

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@soroushm

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@soroushm

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/release.yml:
- Line 79: Quote the entire run command in the release-commit step so the colon
in `chore(release): v$VERSION` is parsed as part of the command rather than YAML
syntax. Preserve the existing `node scripts/release-commit.mjs` invocation and
version interpolation.

In `@README.md`:
- Around line 34-43: Update the default-branded relay logic and the
corresponding README instructions so passing github-token: '' always skips
commenting, including when the relay is otherwise available. Ensure the
short-circuit is applied before relay or fallback token handling, and keep the
documented behavior consistent in both referenced sections.

In `@release-notes/1.1.0.md`:
- Around line 16-24: Update the release notes’ “Changed defaults” guidance to
state that setting github-token: '' alone preserves the previous skip-comment
behavior. Remove the contradictory requirement to also set branded: 'false',
keeping the contract consistent with the README and PR objective.

In `@src/index.ts`:
- Around line 38-55: Enforce the empty github-token opt-out in src/index.ts at
the branded relay flow by skipping relay posting when github-token is empty,
then use the GitHub-token fallback behavior. In action.yml lines 41-44, retain
the empty-token description; update README.md lines 34-43 and 84-85 to document
tokenless usage and the single-input opt-out; remove the contradictory
requirement to also set branded: 'false' from release-notes/1.1.0.md lines
16-24.

In `@src/relay.ts`:
- Around line 23-28: Update postViaRelay’s relay fetch to include an
AbortSignal.timeout with the appropriate request deadline, allowing a stalled
request to reject and trigger the existing github-token fallback. Add or update
tests covering the timeout/abort path and verifying fallback behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 73f03b0e-78bd-444c-aa74-0c7cd9d360c2

📥 Commits

Reviewing files that changed from the base of the PR and between 1732049 and 95cabce.

📒 Files selected for processing (10)
  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • README.md
  • action.yml
  • package.json
  • release-notes/1.1.0.md
  • scripts/release-commit.mjs
  • src/index.ts
  • src/relay.test.ts
  • src/relay.ts

Comment thread .github/workflows/release.yml Outdated
Comment thread README.md
Comment thread release-notes/1.1.0.md Outdated
Comment thread src/index.ts Outdated
Comment thread src/relay.ts
@sonarqubecloud

Copy link
Copy Markdown

@soroushm
soroushm merged commit fe555b7 into main Jul 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Task] Default github-token to the workflow's runtime token

1 participant