Skip to content

Goal 1 — 404-fixture scorecard + v0.5 baseline run #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
k-rutten wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Goal 1 — 404-fixture scorecard + v0.5 baseline run #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7fe96df
Goal 1 — 404 fixture scorecard, fixed before baseline run
k-rutten Jul 27, 2026
c0cf8e9
Goal 1 baseline run (v0.5): O0/O2/O3/O4 pass, O1 fail, O5 not measurable
k-rutten Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions GOALS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Goals — measurable objectives per improvement iteration

> Working method (agreed 2026-07-27): every improvement iteration starts from **jointly agreed, measurable objectives** — extracted from an article, lesson, or dogfood run — before any SKILL.md edit lands. A goal is fixed *before* its first test run; results append to the run log; lessons go to [LESSONS.md](LESSONS.md). Test-emit code itself is throwaway — the scorecard result is the artifact.

---

## Goal 1 — 404 fixture: 0/6 historical failure modes

**Origin:** Orbit (Polar) article mining (2026-07) × LESSONS 2026-05-08 (six failure modes from pre-skill internal testing). The article contributes the method — every objective is scored by a deterministic check, not by self-assessment ("Docs are a suggestion. CI is a contract."). The lessons contribute the test cases — real failures observed on this exact fixture.

**Goal statement:** On the 404-page fixture, an implement-pass with the current skill reproduces **0 of the 6** historically observed failure modes, scored by the deterministic checks below. Objectives O0–O4 gate; O5 is tracked (known-fail pending mapping fluidity-intent, DEFERRED-FIXES 🟢 #6).

**Dropped:** a suppression-tracking objective (Orbit's `eslint-disable`-as-bug) — zero support in LESSONS history; parked as future watch-item.
Comment on lines +7 to +13

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Make the “0/6” objective traceable to six distinct failure modes.

The inventory is not one-to-one: O1 bundles multiple failures, O4 bundles deletion and cache leakage, and O5 represents viewport-fill behavior separately from the six failures listed in the 2026-05-08 confirmation. Add an explicit failure-mode-to-objective mapping, or rename the goal to describe six scorecard objectives rather than six historical failures.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@GOALS.md` around lines 7 - 13, Clarify the Goal 1 “0/6” claim by adding an
explicit mapping from each of the six historical failure modes to the relevant
objectives, accounting for bundled failures in O1 and O4 and O5’s separate
viewport-fill behavior. Alternatively, rename the goal to state that it measures
six scorecard objectives rather than six historical failures.


### Fixture

| Fact | Value |
|---|---|
| Figma node | `14961:2761` — "Pagina niet beschikbaar" (404), file `RFmWghsnS3kQn5fDZwZw58` (eBlinqx-WorQX) |
| Project | `~/Github/WorQX/Figma-to-code/` |
| Mapping | `figma-to-code-mapping/` (tokens.md, components.md, drifts.md, verify-queue.md, 6 per-component specs) |
| MCP cache | `figma-mcp-context/` — **no entry for 14961:2761 at baseline** |
| Codebase | `codebase/worqx-codebase-2026-04-28/` — React + Emotion + MUI theme (`src/theme/tokens.ts`) |
| Pre-existing target | `src/pages/error-pages.tsx` — the file the pre-skill test destroyed |
| Known schema gap | fixture `tokens.md` predates current mapping schema: no `§ Project styling stack`, no `§ Auto-layout conventions` (stack implied in prose). Not pre-fixed — gaps the skill hits are data. |

### Protocol

1. **Phase 0 — unmapped-node halt.** Request emit for `14961:2761` with no mapping present. Scores O0.
2. **Fixture prep (unscored).** Run `figma-to-code-mapping` to map the 404 node (per-component spec + cache entry). This is the pipeline as designed, not part of implement's score.
3. **Phase 1 — mapped emit.** Full implement-pass (B1–B8). Scores O1–O5.
4. **Teardown.** Score, append to run log, write LESSONS entries, `git restore` the WorQX test emit.
Comment on lines +29 to +32

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Pin the fixture state after unscored preparation.

Fixture preparation is declared unscored, yet its mapping and cache artifacts remain permanently. Future runs may therefore use different fixture inputs than v0.5, making score changes indistinguishable from fixture drift. Record a fixture commit/hash or version the prepared snapshot before scoring.

Also applies to: 54-55

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@GOALS.md` around lines 29 - 32, Update the fixture-preparation and scoring
workflow in GOALS.md so the prepared mapping and cache artifacts are pinned
before Phase 1 scoring. Record a fixture commit, hash, or versioned snapshot
after the unscored preparation, and require later runs to use that exact
snapshot so score changes remain comparable.


### Scorecard

| # | Objective | Gate? | Historical failure (LESSONS 2026-05-08) | Deterministic check |
|---|---|---|---|---|
| O0 | Halt on unmapped node — no improvised emit | gate | bare MCP emitted without mapping ground-truth | Phase 0: skill halts + routes to mapping; `git status` in WorQX shows **0 files written** |
| O1 | Zero raw style values | gate | raw CSS props without consulting tokens; random colors where tokens existed | on the emit diff: color literals (`#hex`, `rgb(`, `hsl(`) = **0**; every numeric px literal either matches a `raw, legitimate` verdict in tokens.md or is hoisted — unexplained px = fail |

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Make O1’s check semantic and verdict-aware.

The rule currently fails on every color literal, while the run notes say verdict-backed #F4F7F9 hoists should pass. Conversely, raw values outside the listed #hex/rgb/hsl and px patterns can evade the check. Define the accepted verdict-backed forms and cover all raw-style categories consistently.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@GOALS.md` at line 39, Update the O1 “Zero raw style values” gate in GOALS.md
to be verdict-aware: permit color literals such as `#F4F7F9` only when backed by
an explicit tokens.md verdict and hoisted appropriately, while rejecting
unapproved colors and raw values outside the current `#hex/rgb/hsl` and px
patterns. Define the accepted verdict-backed forms and apply the same semantic
validation consistently across all raw-style categories.

| O2 | Single styling surface | gate | `className` alongside Emotion | on the emit diff: `className=`, `style={{`, Tailwind-utility strings = **0** occurrences |
| O3 | Literal fidelity to mapping | gate | generic "probeer opnieuw" instead of documented copy | every user-facing string in the emit matches spec § Literal strings exactly; mismatches = **0** |
| O4 | Non-destructive, in-scope writes | gate | existing 404 page deleted + replaced; cache file committed to git | `git diff` in WorQX: no deleted exports/lines in pre-existing code beyond the intended edit-range; changed files = intended target(s) only; nothing staged from `figma-mcp-context/` |

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Check staged changes and the canonical cache path.

Plain git diff omits staged changes, so a staged cache file can escape O4. The check also names figma-mcp-context/, while SKILL.md uses figma-context/; the recorded path drift makes this historical failure mode unenforceable. Use a HEAD-relative status/diff check and verify the actual cache path(s).

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@GOALS.md` at line 42, Update the O4 goal in GOALS.md to require HEAD-relative
status and diff checks that include both staged and unstaged changes, and
validate the repository’s actual canonical cache path rather than the stale
figma-mcp-context/ name. Align the path check with the cache path used by
SKILL.md, while preserving the existing safeguards against unintended deletions,
replacements, and changes outside intended targets.

| O5 | Visual fidelity deltas | tracked | emit filled ~40% of viewport instead of 100vw | B8 7-point diff: count ✗/⚠ deltas. Viewport-fill ✗ is **expected** until mapping documents fluidity-intent — track, don't gate |

### Run log

| Date | Skill version | O0 | O1 | O2 | O3 | O4 | O5 (✗/⚠) | Notes |
|---|---|---|---|---|---|---|---|---|
| _pre-skill_ | none (bare MCP / mapping-only) | ✗ | ✗ | ✗ | ✗ | ✗ | n/a | the 2026-05-08 baseline: 6/6 failure modes observed |
| 2026-07-27 | 0.5 | ✓ | ✗ | ✓ | ✓ | ✓ | not measurable | Baseline run. See notes below. |
Comment on lines +43 to +50

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Give non-runnable B8 a deterministic verdict.

O5 is recorded as “not measurable,” but the protocol requires deterministic scoring and does not define whether this status is excluded, failed, or passed. Adopt the lesson’s explicit not executable — <reason> result together with a fixed scoring policy or measurable fallback.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@GOALS.md` around lines 43 - 50, Update the O5 entry in the B8 run log and its
scoring guidance to use the explicit “not executable — <reason>” result, with a
deterministic policy stating whether that outcome is excluded, failed, or
passed. If O5 remains executable, define and apply a measurable fallback instead
of recording “not measurable.”


### Run notes — 2026-07-27 baseline (v0.5)

- **O0 ✓** — B1 halted on the unmapped node, routed to mapping, 0 files written. Fixture prep then ran the mapping skill (error-page spec + cache + 3 new drifts + 3 verify items — kept as permanent fixture improvement).
- **O1 ✗** — two components: (a) *scorecard calibration*: 1 color literal `#F4F7F9`, but hoisted + verdict-backed (`token-mismatch` drift documented) — the O1 color check should allow verdict-backed hoists, mirroring the px rule; (b) *genuine gap*: ~50 inline px literals for illustration/absolute geometry (448.5px, 987px, layer positions) with no hoist path and no verdict home — rule #2 covers style tokens, but decorative geometry has no rule. Substantive fail.
- **O2 ✓** — 0 second-styling-API occurrences (pure Emotion; Figma MCP's Tailwind reference output fully translated).
- **O3 ✓** — all 4 literal strings exact match with spec § Literal strings.
- **O4 ✓** — only intended files touched (error-pages.tsx edit + co-located styles file per loading-page.styles.ts precedent); BrokenPage/NoAccessPage/UnwelcomeBanner intact; no cache files staged.
- **O5 not measurable** — fixture codebase is a source-only snapshot (no package.json/deps): B8.1–B8.5 cannot run. Neither can any machine contract check (no lint/typecheck available). Fixture limitation to fix or accept.
- **Halt-behavior observed:** B6 check #6 (verify-queue items on same spec) and check #7 (Critical redesign drift) both fired correctly — real runs wait for the user there; test proceeded under standing test authorization as the update-code decision.
- **Schema drift fixture ↔ skills** (surfaced for free): cache folder `figma-mcp-context/` vs skill's `figma-context/`; tokens.md 4-column format without verdict column; specs in `figma-to-code-mapping/components/` not co-located; cache git-tracked while mapping setup mandates gitignore.
15 changes: 15 additions & 0 deletions LESSONS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,18 @@ Proposal: Dogfood v0.4 on a real implement-pass before adding v0.5 (design-fidel
Situation: v0.4 audit-fixes (B7 read-before-write, rule #3 binary halt, B8 active diff) sluit niet de drift loop-closing gap — drifts blijven in mapping's `drifts.md` archief zonder structureel beslispunt. Architectuur-analyse toont: detectie + logging klaar, surfacing aan beslisser + decision-routing + status-update ontbreken.
What worked: Rule #8 uitgebreid naar drift-summary in chat na emit met decision-prompt per drift. Rule #10 uitzondering voor `drifts-mapping.md` (implement-owned, parallel aan mapping's `drifts.md`). § Write contract uitgebreid. B8.5 prompt + nieuwe B8.6 drift-summary step. Implement is self-sufficient — geen mapping-coordination nodig voor v0.5 ship.
Proposal: Bump v0.4 → v0.5 met loop-closing erin. Originally v1.0 scope, maar onafhankelijk van mapping coordination via aparte file. Dogfood-pass valideert of de loop in praktijk werkt.

[LESSON — 2026-07-27] [confirmation]
Situation: Eerste echte dogfood-run (GOALS.md Goal 1): volledige emit-pass op het WorQX 404-fixture met v0.5, gescoord op scorecard O0-O5.
What worked: O0/O2/O3/O4 ✓ — halt op unmapped node, single styling API, exacte literal strings, non-destructieve in-scope edit; vier van de zes 2026-05-08 failure modes aantoonbaar afgedekt onder echte emit-condities. B6-halts (verify-queue + Critical drift) vuurden correct.

@coderabbitai coderabbitai Bot Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use consistent English wording.

“B6-halts … vuurden correct” mixes Dutch into the surrounding English prose. Replace it with “fired correctly” for consistent documentation.

🧰 Tools
🪛 LanguageTool

[grammar] ~31-~31: Ensure spelling is correct
Context: ...6-halts (verify-queue + Critical drift) vuurden correct. Proposal: Rule-set houden; sco...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@LESSONS.md` at line 31, Update the B6-halts sentence in the documented “What
worked” entry to replace the Dutch wording “vuurden correct” with the English
phrase “fired correctly,” leaving the rest of the documentation unchanged.

Source: Linters/SAST tools

Proposal: Rule-set houden; scorecard-runs herhalen per versie-bump (run log in GOALS.md).

[LESSON — 2026-07-27] [correction]
Situation: O1 faalde in de baseline-run: illustratie/absolute-geometrie emitteerde ~50 inline px-literals (layer-posities, 448.5px, 987px) — geen hoist-pad, geen verdict-thuis; rule #2 dekt alleen stijl-tokens.
What did not work: Emit-discipline heeft geen regel voor decoratieve geometrie; daarnaast telt de O1-kleurcheck verdict-backed hoists (#F4F7F9 met token-mismatch-drift) onterecht als fail — scorecard-calibratie nodig.
Proposal: Rule #2 verhelderen (of mapping-side illustratie-geometrie-conventie) + O1-kleurcheck gelijktrekken met de px-regel (verdict-backed hoist = pass).

[LESSON — 2026-07-27] [correction]
Situation: B8 was in de baseline-run dead-on-arrival: fixture-codebase is een source-only snapshot (geen package.json/deps) — B8.1-B8.5 én elke lint/typecheck onmogelijk; O5 unmeasurable.
What did not work: B8 veronderstelt stilzwijgend een runnable project en degradeert zonder expliciet verdict; ook de geplande v0.6 B8.0 machine-contract check heeft een gedefinieerde fallback nodig ("geen tooling beschikbaar" = expliciete uitkomst, geen stille skip).
Proposal: B8 een expliciet "not executable — reden" verdict geven; fixture runnable maken of O5 als known-unmeasurable accepteren.