Skip to content

Close the M2 loop: a run opens a pull request, and /fix lands on it - #132

Merged
artyomsv merged 6 commits into
masterfrom
feat/factory-pull-requests
Sep 10, 2026
Merged

Close the M2 loop: a run opens a pull request, and /fix lands on it#132
artyomsv merged 6 commits into
masterfrom
feat/factory-pull-requests

Conversation

@artyomsv

Copy link
Copy Markdown
Owner

What

The M2 loop ran end to end for the first time, and this is what it took. Everything here was
found by running it against a real forge and a real model, not by reading the code.

docs/UNVERIFIED.md said the loop was "covered in three places and joined in none". It is joined
now, and three of the four things that stopped it were defects nothing had ever executed.

The three fixes

1. Codex never started. codex-cli 0.146.0 does not read OPENAI_API_KEY: with the variable
set and no ~/.codex/auth.json it exits 1 at once, printing No such file or directory (os error 2)
and no NDJSON. Every run this deployment dispatched died in about thirty seconds and was reported as
MODEL_UNAVAILABLE, because "exited non-zero and said nothing" was the only reading the classifier
had — so the diagnosis went to the model and to the key while the CLI had never run. The arm now
performs the login OpenAI documents for programmatic use, piping the key from printenv so argv
still carries no secret, and a failed login exits with a code of its own and is reported as an agent
failure rather than as a model that would not answer.

2. Nothing opened a pull request. M2 shipped the PullRequestSink port, three forge adapters
and the body builder, and no caller — pullRequestSink(...) was reachable from nothing. The
milestone's own goal, "a run can end at a pull request rather than at a branch", was true of the
parts and false of the product. The result saga now proposes what a build run pushed:

  • Last, and after the projection, because it reads the row the projection has just written.
  • A fix run proposes nothing: its change is already on a pull request's own source branch
    (ADR-040) and already under review.
  • The forge is asked for an existing request for that head before one is opened, so a redelivered
    result cannot propose twice — and the run's own number is checked first so it does not even ask.
  • Opening happens after the push succeeded and can fail on its own, so a failure is recorded
    beside the run
    and never changes its status. The work is on the remote either way, and throwing
    would nack the record and re-run every consumer that already applied it.
  • V57 adds the four columns this needs. The task line a title needs is stored at dispatch,
    because the prompt is not a column and a finished run knows only a branch and a list of paths.

3. wasNull() answered about the wrong column. The plan read pr_number, then built its record
— and the argument list read three more columns before asking wasNull(), which reports on the last
column read. It answered about the task summary, which is present on every dispatched run, so a
queued row came back with pr_number = 0. Zero is not null, "already proposed" was therefore true,
and the opener returned with no log line, no error and no row. The first live run after the wiring
landed pushed a branch that nothing proposed, silently. The projection test now queues a row with
a summary and asserts the number is absent, which is the pairing that makes the trap visible.

What ran, in order

Step Result
POST /api/runs → agent → push branch spire/issue-28-5 as the FACTORY account
Pull request opened by the run #29
Webhook → review of that pull request completed, by the REVIEWER account
A planted BLOCKER on a second pull request found by the reviewer unaided
/fix on that finding → run → push onto the pull request's own branch, no second request

ADR-038's separation held throughout: the reviewer is one account, the factory another, and the
run id of a fix is the finding's own comment id.

Also here

  • docs/superpowers/specs/2026-09-10-accounts-scopes-and-roles-analysis.md — the analysis of the
    duplicated credentials on the Accounts and Context screens, per-provider scope introspection with
    sources and dates (GitHub fine-grained tokens report nothing; Atlassian classic tokens have no
    scopes at all), why ADR-038 stands unamended, and two options with a recommendation.
  • docs/factory/ROADMAP.md M3 — the authorisation rule the first live /fix argued for:
    authorise on write access to the repository, not on authorship, because "the pull request's
    author" on a public repository means any stranger.

Verification

  • testFast --rerun-tasks green; spire-orchestrator and spire-harness-codex green (1210 + 48).
  • Mutation-verified: the login step, the login-failure classification, the shell quoting guard, the
    BUILD-only rule, the already-proposed guard, the find-first, the never-fail-the-run catch, the
    saga's ordering, and the wasNull read — each broken on purpose, each killed by its own test.
  • Rebased onto master after Renew OIDC sessions instead of re-authenticating the browser #131; no file in this branch touches that work.

Not fixed here

A refused /fix is logged and posts no reply to the thread, so from the forge's side it looks
ignored. Found while running this; it belongs with M3's actor allowlist rather than with the wiring.

Codex CLI 0.146 does not read OPENAI_API_KEY. With the variable set and no
~/.codex/auth.json it exits 1 at once, printing "No such file or directory"
and no NDJSON, so every run this deployment dispatched died in about thirty
seconds. The classifier then called it MODEL_UNAVAILABLE, because "exited
non-zero and said nothing" was the only reading it had, and the diagnosis
went to the model and the key while the CLI had never started.

The arm now performs the login OpenAI documents for programmatic use,
piping the key from printenv so argv still carries no secret, and a login
that fails exits with a code of its own that is reported as an agent
failure rather than as a model that would not answer. The invocation is
one shell line because --with-api-key reads the key from stdin and nothing
else avoids the pipe; exec replaces the shell, and the two values
interpolated into it are quoted and refused if they could close the quote.
M2 shipped the PullRequestSink port, three forge adapters and the body
builder, and no caller: pullRequestSink() was reachable from nothing, so
every run ended at a pushed branch. The first live run proved it — the
work was correct, the branch landed, and the milestone's own goal, that a
run can end at a pull request, was true of the parts and false of the
product.

The result saga now proposes what a build run pushed, last and after the
projection, because it reads the row the projection has just written. A
fix run proposes nothing: its change is already on a pull request's own
source branch and already under review. The forge is asked for an
existing request for that head before one is opened, so a redelivered
result cannot propose twice, and the run's own number is checked first so
it does not even ask.

Opening happens after the push succeeded and can fail on its own, so a
failure is recorded beside the run and never changes its status: the work
is on the remote either way, and throwing here would nack the record and
re-run every consumer that already applied it. The task line a title
needs is stored at dispatch, since the prompt is not a column and a
finished run knows only a branch and a list of paths.
The pull-request plan read the number, then built its record — and the
argument list read three more columns before asking wasNull(), which
reports on the LAST column read. It answered about the task summary, and
a summary is present on every dispatched run, so a queued row came back
with pr_number = 0. Zero is not null, "already proposed" was therefore
true, and the opener returned without a log line, an error or a row.

The first live run after the wiring landed pushed a branch that nothing
proposed, and nothing anywhere said why. The projection test now queues a
row WITH a summary and asserts the number is absent, which is the pairing
that makes the trap visible; putting the old line back fails it.

@artyomsv artyomsv left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changes at 7e6a42f. Three actionable findings are attached inline: the shell invocation breaks two existing worker tests, proposal outcomes are unavailable through the runs API/UI, and a re-armed dispatch can propose its new work with the previous prompt's summary.

Validation: inspected the dispatch/result/projection/forge paths and changed tests; checked the current CI logs (fast tests pass; service tests fail in two RunUnitBuilderTest cases); reproduced the stale-summary retry against an isolated PostgreSQL 17 instance using the actual queued() SQL extracted from this commit. The reproduction accepted the retry with Task B while returning Task A as task_summary. I did not rerun the full Java suite locally. The temporary database was removed and the worktree is unchanged.

Submitted as a COMMENT review because the authenticated GitHub account is the PR author; GitHub does not permit requesting changes on one's own PR.

…n screen

Three findings from the review of this branch.

The codex arm's argv became a shell line, and two RunUnitBuilderTest cases
still asserted the old shape — the failures in the service-test job. They
now read the line rather than its elements, keeping what they were about:
the workspace and the harness are named in it, the prompt is not, and the
prompt position is still last.

A re-armed dispatch kept the first prompt's task summary while executing
the retry's prompt, so the pull request the successful run opened would
have been titled with work nobody asked for. Reproduced in review against
Postgres with the real upsert. The summary now follows the retry. It is
deliberately not part of the identity comparison: the prompt never has
been, so comparing its summary would refuse the ordinary "fix the prompt
and retry", and what remains true is written down — a re-arm can change
what the run does.

A proposal that failed was recorded and shown nowhere: both runs
endpoints and the dashboard read the same whether a pull request opened
or the forge refused it, and the run stays succeeded either way because
the branch is on the remote. The list and the detail carry the URL and
the reason now, and the runs screen links the request or says it was not
proposed with the reason on the hover. The cost test that asserted "some
dash exists" is scoped to the cost cell, since the new column renders one
too — the third time that assertion has been satisfied by an empty
neighbour.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant