Skip to content

fix(e2e): per-run databases + specs rewritten against the engine - #13

Merged
adbarc92 merged 1 commit into
mainfrom
fix/e2e-per-run-db
Jul 25, 2026
Merged

fix(e2e): per-run databases + specs rewritten against the engine#13
adbarc92 merged 1 commit into
mainfrom
fix/e2e-per-run-db

Conversation

@adbarc92

Copy link
Copy Markdown
Owner

Closes the last known functional defect blocking the v1.0.0 release. It turned out to be two independent problems, and the handoff misattributed which one was red.

What actually failed the e2e job

Not database is locked. All four Playwright specs were last touched 2026-06-02 and assert on the retired LangGraph chat flow — every failure in the CI log is getByText(/Clarifying question #1/) → "element(s) not found".

The engine runs the clarify Q&A loop inside the worker (product_owner auto-answers clarifying_pm, agent_adapter.py::_run_clarify_loop), so the browser goes straight from "Start" to the PRD gate and those chat turns never exist. The database is locked tracebacks in the same log were real, but they were concurrent server-side noise, not the failing assertion.

Specs rewritten, not retired

The handoff flagged retiring the job as a user-facing call. It isn't needed — the engine's flow is fully drivable through the UI, so retiring would have discarded real coverage.

new spec covers
clarify-gate.spec.ts gate opens with the PRD rendered · rejection loop re-runs the phase · reload rehydrates from the engine snapshot
full-run.spec.ts both gates approved, all six phases run to completion

The lock defect, confirmed and fixed

backend/main.py passed a constant data/web.db into every start_run, and each start_run boots its own state-server process. The store's single-writer guarantee is an asyncio.Lock — it serialises nothing across processes, so two live runs put two OS writers on one SQLite file.

Each run now takes a unique sibling of APPFORGE_WEB_DB. That is what makes the README's load-bearing "single SQLite writer" claim actually true, rather than true-only-while-one-run-is-alive — which is why this was preferred over serialising runs.

Evidence (A/B, both directions)

Engine level, 4 concurrent runs, gates driven:

outcome
shared DB (before) 4/4 runs failed, 2 workers killed by database is locked
per-run DBs (after) 4/4 done, 13/13 tasks each, 0 lock deaths

Browser level, this same suite: without the fix it passes but logs 3 lock-induced worker crashes; with it, 0. Contention also launders lock errors into task retries that can fail a run outright, so the unfixed build was a live flake source for the new suite.

Also ruled out, as the handoff advised: busy_timeout is not involved — the failing path is a cross-process write conflict, not a timeout.

Verification

All run with data/ absent (mv data data.bak), since a stale local data/ is exactly how the 7bfa00d bug reached main.

  • uv run pytest tests/156 passed (was 155), coverage 86.99% (gate 70%)
  • cd e2e && npx playwright test4 passed in ~1.1m, 0 lock errors
  • cd frontend && npm test28 passed
  • uv run ruff check backend/ tests/ · uv run black --check backend/ tests/ — clean

Notes / deliberate choices

  • Per-run DB files are not deleted on teardown. data/ is gitignored scratch and a per-project database is useful for post-hoc inspection, so deleting was the more destructive option. They do accumulate over a long local session — logged as a known gap in docs/STATUS.md.
  • Pre-existing, out of scope: at the design gate the approval card renders no body, because PlanViewer reads adr/tasks/design_spec from the store and only the reload (project_state) path populates them. Reloading shows it. Logged, not touched.

…the engine

The red `e2e` job was two independent defects, not one.

What actually failed the job: all four Playwright specs predated the engine
rewrite and asserted on the retired LangGraph chat flow. Every failure in the
CI log is `getByText(/Clarifying question #1/)` -> "element(s) not found". The
engine runs the clarify Q&A loop inside the worker (product_owner auto-answers
clarifying_pm), so the browser goes straight from "Start" to the PRD gate and
those chat turns never exist. The `database is locked` tracebacks in the same
log were real, but they were server-side noise rather than the failing
assertion.

The specs are rewritten rather than retired -- the engine's flow is fully
drivable through the UI, so retiring the job would have discarded real
coverage. clarify-gate.spec.ts covers the gate opening with the PRD rendered,
the rejection loop re-running the phase, and reload rehydration from the engine
snapshot; full-run.spec.ts approves both gates and drives all six phases to
completion.

The lock defect is separately real and also fixed. backend/main.py passed a
constant `data/web.db` into every start_run, and each start_run boots its own
state-server PROCESS. The store's single-writer guarantee is an asyncio.Lock,
which serialises nothing across processes, so two live runs put two OS writers
on one SQLite file. Each run now takes a unique sibling of APPFORGE_WEB_DB,
which is what makes the README's load-bearing "single SQLite writer" claim
actually true rather than true-only-while-one-run-is-alive.

Evidence, A/B in both directions. Engine level, 4 concurrent runs: shared DB ->
4/4 runs failed with 2 workers killed by `database is locked`; per-run DBs ->
4/4 done, 13/13 tasks each, 0 lock deaths. Browser level, same suite: without
the fix it passes but with 3 lock-induced worker crashes; with it, 0. The
contention also launders lock errors into task retries that can fail a run
outright, so the unfixed build was a live flake source for the new suite.

Backend suite 155 -> 156, coverage 86.99%. Verified with data/ absent.
@adbarc92
adbarc92 merged commit d24ccab into main Jul 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant