diff --git a/.gitignore b/.gitignore index 630c23b7..d1f24c9f 100644 --- a/.gitignore +++ b/.gitignore @@ -481,6 +481,13 @@ FodyWeavers.xsd # 10's Native AOT publish-and-run proof) - regenerated by that script, never committed. .local-nuget-feed-nunit-aot-smoke/ +# Raw per-run outputs from skills/compono-workspace/ eval iterations - transient, regenerated by +# each run. grading.json/timing.json/benchmark.json/feedback.json ARE committed (this repo's own +# established evaluation-evidence convention, see skills/compono-workspace/benchmarks/), only the +# bulky raw outputs/ directories are excluded. +skills/compono-workspace/iteration-*/**/outputs/ +skills/compono-workspace/skill-snapshot/ + # macOS .DS_Store .DS_Store? diff --git a/README.md b/README.md index e0b17ae2..5263a9aa 100644 --- a/README.md +++ b/README.md @@ -128,8 +128,11 @@ instead of opening a public issue. ## Status -Compono is under active development. APIs are experimental until the -first public preview. +Compono is publicly released, with stable `0.x` packages on NuGet. It's +still pre-`1.0`, so the public API can still change before the `1.0` +compatibility boundary — see +[Installation](https://layeredcraft.github.io/compono/getting-started/installation/) +for exact package versions. ## License diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 3c4855a8..2c9fe125 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -7,14 +7,14 @@ composed test method parameters/theories. That's two packages — `Compono` plus whichever test-framework integration matches your test host: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.XunitV3 --prerelease +dotnet add package Compono +dotnet add package Compono.XunitV3 # or, for TUnit: -dotnet add package Compono.TUnit --prerelease +dotnet add package Compono.TUnit # or, for MSTest (MSTest.TestFramework 4.0.0+ - see the Compono.MSTest Package Guide): -dotnet add package Compono.MSTest --prerelease +dotnet add package Compono.MSTest # or, for NUnit (NUnit 3.14.0+ - see the Compono.NUnit Package Guide): -dotnet add package Compono.NUnit --prerelease +dotnet add package Compono.NUnit ``` This tutorial's assertions (`.Should()`, throughout this site's own @@ -29,17 +29,15 @@ dotnet add package AwesomeAssertions Add the rest of the ecosystem as your tests need it: ```bash -dotnet add package Compono.NSubstitute --prerelease # automatic substitute composition -dotnet add package Compono.Bogus --prerelease # semantic fake data (names, emails, ...) -dotnet add package Compono.DependencyInjection --prerelease # row.AsServiceProvider() bridge -dotnet add package Compono.Http --prerelease # TestHttpHandler for HttpClient tests +dotnet add package Compono.NSubstitute # automatic substitute composition +dotnet add package Compono.Bogus # semantic fake data (names, emails, ...) +dotnet add package Compono.DependencyInjection # row.AsServiceProvider() bridge +dotnet add package Compono.Http # TestHttpHandler for HttpClient tests ``` -Every package targets `net8.0`/`net9.0`/`net10.0`/`net11.0`. Until the first stable `1.0` -release, every published version is a `0.x.y-preview.N` prerelease — the -`--prerelease` flag (or an explicit ``) -is required for `dotnet add package`/NuGet restore to pick it up at all, -since a plain `dotnet add package` skips prerelease versions by default. See +Every package targets `net8.0`/`net9.0`/`net10.0`/`net11.0` and has a +stable release, so a plain `dotnet add package` picks up the right version +for all of them with no extra flag. See [Package Guides](../packages/index.md) for what each package is for and when to add it. diff --git a/docs/migrating-from-autofixture.md b/docs/migrating-from-autofixture.md index dbf34b36..352c2dce 100644 --- a/docs/migrating-from-autofixture.md +++ b/docs/migrating-from-autofixture.md @@ -67,17 +67,16 @@ AutoFixture users migrating xUnit tests need: anonymous values. ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.XunitV3 --prerelease -dotnet add package Compono.NSubstitute --prerelease -dotnet add package Compono.Bogus --prerelease +dotnet add package Compono +dotnet add package Compono.XunitV3 +dotnet add package Compono.NSubstitute +dotnet add package Compono.Bogus ``` Install matching versions of every Compono package you add — mixing versions across packages isn't supported; see [Package Guides: Version Compatibility](packages/index.md#version-compatibility). -See [Installation](getting-started/installation.md) for the full setup, -including why `--prerelease` is required during public preview. +See [Installation](getting-started/installation.md) for the full setup. ## Quick concept map diff --git a/docs/packages/compono-bogus.md b/docs/packages/compono-bogus.md index 4f1352c3..9bbd3894 100644 --- a/docs/packages/compono-bogus.md +++ b/docs/packages/compono-bogus.md @@ -12,8 +12,8 @@ You want composed string/data members to look like plausible real values placeholder strings: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.Bogus --prerelease +dotnet add package Compono +dotnet add package Compono.Bogus ``` If anonymous placeholder values are fine for your tests, you don't need diff --git a/docs/packages/compono-dependencyinjection.md b/docs/packages/compono-dependencyinjection.md index 7aaab599..af49b232 100644 --- a/docs/packages/compono-dependencyinjection.md +++ b/docs/packages/compono-dependencyinjection.md @@ -19,8 +19,8 @@ to enumerate and manually register every dependency a system under test might ask for: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.DependencyInjection --prerelease +dotnet add package Compono +dotnet add package Compono.DependencyInjection ``` This package is deliberately **not** framework-specific. It doesn't diff --git a/docs/packages/compono-http.md b/docs/packages/compono-http.md index 7ca645d5..51b4fa3b 100644 --- a/docs/packages/compono-http.md +++ b/docs/packages/compono-http.md @@ -18,8 +18,8 @@ pipeline against a configured HTTP response — not substitute an application-level interface: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.Http --prerelease +dotnet add package Compono +dotnet add package Compono.Http ``` If the seam under test is already an ordinary application interface diff --git a/docs/packages/compono-logging.md b/docs/packages/compono-logging.md index a9cacdc2..7491a405 100644 --- a/docs/packages/compono-logging.md +++ b/docs/packages/compono-logging.md @@ -19,8 +19,8 @@ message, structured properties, exception, scope — not substitute an unrelated application interface: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.Logging --prerelease +dotnet add package Compono +dotnet add package Compono.Logging ``` This package depends only on `Compono` and diff --git a/docs/packages/compono-mstest.md b/docs/packages/compono-mstest.md index 387d8637..40ed8717 100644 --- a/docs/packages/compono-mstest.md +++ b/docs/packages/compono-mstest.md @@ -9,8 +9,8 @@ that composes test method parameters directly, instead of hand-building You write MSTest tests and want method parameters composed automatically: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.MSTest --prerelease +dotnet add package Compono +dotnet add package Compono.MSTest ``` `Compono.MSTest` doesn't add an MSTest test host for you — it integrates diff --git a/docs/packages/compono-nsubstitute.md b/docs/packages/compono-nsubstitute.md index c97f05a7..22ff556f 100644 --- a/docs/packages/compono-nsubstitute.md +++ b/docs/packages/compono-nsubstitute.md @@ -12,8 +12,8 @@ abstract classes) that you'd otherwise create with `Substitute.For()` by hand: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.NSubstitute --prerelease +dotnet add package Compono +dotnet add package Compono.NSubstitute ``` If none of your composed types have interface/delegate/abstract-class diff --git a/docs/packages/compono-nunit.md b/docs/packages/compono-nunit.md index 86170c8e..4ff8943a 100644 --- a/docs/packages/compono-nunit.md +++ b/docs/packages/compono-nunit.md @@ -9,8 +9,8 @@ attribute that composes test method parameters directly, without needing You write NUnit tests and want method parameters composed automatically: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.NUnit --prerelease +dotnet add package Compono +dotnet add package Compono.NUnit ``` `Compono.NUnit` doesn't add an NUnit test host for you — it integrates with diff --git a/docs/packages/compono-testdoubles.md b/docs/packages/compono-testdoubles.md index 2a65abe8..f44d96e6 100644 --- a/docs/packages/compono-testdoubles.md +++ b/docs/packages/compono-testdoubles.md @@ -11,8 +11,8 @@ generated double, without pulling in `Compono.NSubstitute`'s runtime proxy dependency (or when you need the composed path to survive `PublishAot`): ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.TestDoubles --prerelease +dotnet add package Compono +dotnet add package Compono.TestDoubles ``` `Compono.TestDoubles` is not a general-purpose mocking framework — see diff --git a/docs/packages/compono-tunit.md b/docs/packages/compono-tunit.md index 9438ebd4..03e82553 100644 --- a/docs/packages/compono-tunit.md +++ b/docs/packages/compono-tunit.md @@ -9,8 +9,8 @@ custom data source generator. You write TUnit tests and want method parameters composed automatically: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.TUnit --prerelease +dotnet add package Compono +dotnet add package Compono.TUnit ``` `Compono.TUnit` doesn't add a TUnit test host for you — it integrates with diff --git a/docs/packages/compono-xunitv3.md b/docs/packages/compono-xunitv3.md index 8cc93d03..3e6fdebc 100644 --- a/docs/packages/compono-xunitv3.md +++ b/docs/packages/compono-xunitv3.md @@ -10,8 +10,8 @@ You write xUnit v3 tests (`xunit.v3` + the Microsoft Testing Platform runner) and want theory parameters composed automatically: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.XunitV3 --prerelease +dotnet add package Compono +dotnet add package Compono.XunitV3 ``` `Compono.XunitV3` doesn't add an xUnit v3 test host for you — it integrates diff --git a/docs/packages/compono.md b/docs/packages/compono.md index 87448e2b..1a427e4f 100644 --- a/docs/packages/compono.md +++ b/docs/packages/compono.md @@ -15,7 +15,7 @@ dependency of `Compono.XunitV3`/`Compono.NSubstitute`/`Compono.Bogus`, so directly. ```bash -dotnet add package Compono --prerelease +dotnet add package Compono ``` ## What it gives you diff --git a/docs/packages/index.md b/docs/packages/index.md index db77dade..ee014124 100644 --- a/docs/packages/index.md +++ b/docs/packages/index.md @@ -18,10 +18,9 @@ need the first two. | [`Compono.Http`](compono-http.md) | `TestHttpHandler` — a reflection-free `HttpMessageHandler` test double: `OnGet`/`OnPost`/etc. + `When(...)` matching, strict unmatched-request behavior, registration-handle verification. | Your test needs to exercise the real `HttpClient` pipeline against a configured HTTP response, instead of substituting an application-level interface. | | [`Compono.Logging`](compono-logging.md) | `UseLogging()` — `ILogger`/`ILogger` compose as a hand-written `CapturingLogger`/`CapturingLogger`, with structured-property extraction, real scope tracking, and `Verify()` verification. Generation is on by default once installed. | Your composed type takes an `ILogger`/`ILogger` dependency and the test wants to assert what was logged. | -Every package targets `net8.0`/`net9.0`/`net10.0`/`net11.0` and, until the -first stable `1.0` release, publishes as a `0.x.y-preview.N` prerelease — -see [Installation](../getting-started/installation.md) for the exact -`dotnet add package` commands and why `--prerelease` is required. See +Every package targets `net8.0`/`net9.0`/`net10.0`/`net11.0` and has a +stable release — see [Installation](../getting-started/installation.md) +for the exact `dotnet add package` commands. See [ADR-0038](../adr/0038-net8-net9-explicit-multi-target.md) for why `net8.0`/ `net9.0` were added alongside the existing `net10.0`/`net11.0` window. @@ -38,14 +37,14 @@ integration matches your test host — `Compono.XunitV3` for xUnit v3, for NUnit: ```bash -dotnet add package Compono --prerelease -dotnet add package Compono.XunitV3 --prerelease +dotnet add package Compono +dotnet add package Compono.XunitV3 # or, for TUnit: -dotnet add package Compono.TUnit --prerelease +dotnet add package Compono.TUnit # or, for MSTest: -dotnet add package Compono.MSTest --prerelease +dotnet add package Compono.MSTest # or, for NUnit: -dotnet add package Compono.NUnit --prerelease +dotnet add package Compono.NUnit ``` Add `Compono.NSubstitute` and/or `Compono.Bogus` independently, as your diff --git a/docs/research/0022-1-0-readiness-adversarial-review.md b/docs/research/0022-1-0-readiness-adversarial-review.md new file mode 100644 index 00000000..406ef2f8 --- /dev/null +++ b/docs/research/0022-1-0-readiness-adversarial-review.md @@ -0,0 +1,240 @@ +# [RESEARCH-0022] 1.0 Readiness Adversarial Review + +**Status:** Complete. No product/API-design blocker found. Five +release-mechanics/hardening items accepted as required pre-1.0 work +(tracked below, resolved and verified across two follow-up rounds); five +further items explicitly considered and rejected as unnecessary gates. + +## Why this exists + +Final release-readiness gate for Compono 1.0, run as an adversarial +review whose job was to actively try to prove Compono should **not** ship +1.0 yet — not to summarize prior plans' conclusions. Covered: public API +freeze, deferred pre/post-1.0 decisions (disposal, async composition, +framework-binder duplication), source-generation/runtime contracts, +Native AOT/trimming, package correctness (all 11 publishable packages), +framework integrations, canonical samples, documentation sync, CI/package +validation gates, release/versioning readiness, and dogfooding. + +## Method + +Direct repository investigation plus four parallel research forks +(public-API/generator-contract audit; package-correctness/release +audit; framework-integration/canonical-sample audit; AOT/CI-gate audit), +cross-checked against a fresh clean build, full test run, and two real +dogfood-consumer runs (a live external consumer, `trivia-platform`, +restored against freshly-packed current-`HEAD` Compono packages via +`scripts/dogfood-validate.sh`). Three of the four forks were lost +mid-run to a session rate limit; their load-bearing findings were +recovered by direct inspection rather than re-run (see the original +review's own "checks not executed" section for the exact gaps). + +## Original findings (as reported, unmodified) + +**No BLOCKER found under the review's own criteria** (a finding blocks +1.0 only if fixing it after 1.0 likely requires a breaking change, is a +correctness defect in a supported scenario, is an unstable +generated-code/runtime contract, misstates a package's compatibility +contract, or is a genuine trimming/AOT gap where Compono claims +compatibility). One process-integrity finding was flagged at +blocker-equivalent severity because it undermines confidence in every +other gate the review relied on: + +- **[F0] Required status checks on `main` covered only `build / build`** + (blocker-equivalent, not a numbered Release Task) — `Package + Validation` and `AOT Validation` (the gates PLAN-0061/0062 built + specifically for the 1.0 boundary) were not required checks, so either + could fail red and a PR could still merge. + +Six items were originally classified as RELEASE TASKS, numbered F1-F6 +below (this numbering is this record's own stable identifier scheme, not +part of the original review's prose — introduced to remove the ambiguity +a reviewer flagged in an earlier draft that referred back to items only +as "Item N above"): + +1. **[F1]** `breaking-change` label auto-resolves to `v1.0.0` with no + separate decision point (ADR-0031 Amendment 5, already deliberately + wired). +2. **[F2]** Packed-consumer smoke-test coverage in + `package-validation.yaml` covers only 5 of 11 packages + (XunitV3/TUnit/MSTest/NUnit/TestDoubles); `Compono`, `Http`, `Logging`, + `DependencyInjection`, `NSubstitute`, `Bogus` rely on static nuspec + inspection + AOT-smoke packing only. +3. **[F3]** Install docs universally instruct `--prerelease`. +4. **[F4]** ADR-0031's "rolling two-TFM window" prose appeared to + disagree with the actual four-TFM (`net8.0;net9.0;net10.0;net11.0`) + build matrix. +5. **[F5]** `scripts/dogfood-validate.sh` has an undocumented Docker + prerequisite (its default consumer's repository tests use + Testcontainers). +6. **[F6]** 20 xUnit-analyzer warnings (`xUnit1031`/`xUnit1051`) in + `Compono.DependencyInjection.Tests` aren't gated as errors. + +Deferred decisions re-verified safely post-1.0, unchanged from prior +research: disposal/lifetime ownership ([RESEARCH-0015](0015-disposal-ownership-research.md), +Outcome C), async composition ([RESEARCH-0016](0016-async-composition-viability-research.md), +Outcome C), framework-binder duplication ([RESEARCH-0019](0019-framework-binder-duplication-spike-scope.md), +still scoped-not-executed, no second drift bug found on independent +re-check of the negative-seed guard across all four binders). + +## Disposition after product-owner review + +The product owner accepted the original review's core conclusion (no +product/API-design blocker) without reopening it, then narrowed the six +Release Task items: + +**Accepted as required pre-1.0 work (4):** + +- **[F0]** (blocker-equivalent) — required checks. **Resolved and + verified live**: `LayeredCraft/.github` PR #5 (merged) opts Compono out + of safe-settings' `required_status_checks.contexts` management via the + same `{{EXTERNALLY_DEFINED}}` marker `dynamodb-efcore-provider`/ + `devops-templates` already use — confirmed via + `gh api repos/LayeredCraft/compono/branches/main/protection`, before + (`["build / build"]` only, unaffected by the merge) and after applying + a narrow `PATCH .../branches/main/protection/required_status_checks` + (not a full protection-object replace) setting + `["build / build", "package-validation", "aot-gate"]` with + `strict: true`. Re-read after the change: exactly those three contexts + present, no literal `{{EXTERNALLY_DEFINED}}` check registered, and every + other branch-protection field (reviews, `enforce_admins`, linear + history, force-push/delete, conversation resolution, signatures) + byte-identical to before. +- **[F3]** — stale `--prerelease` install guidance. Resolved across three + passes: + 1. First pass: audited and corrected `docs/getting-started/installation.md`, + all 11 `docs/packages/*.md` guides, `docs/packages/index.md`, + `docs/migrating-from-autofixture.md`, and `docs/troubleshooting/faq.md`. + Correction: **most packages already have a stable `0.9.0` release** + on nuget.org (verified live) — `--prerelease` was already unnecessary + and wrong for `Compono`/`Compono.XunitV3`/`Compono.TUnit`/ + `Compono.NSubstitute`/`Compono.Bogus`/`Compono.DependencyInjection`/ + `Compono.Http`/`Compono.Logging`/`Compono.TestDoubles` *today*, not + merely "once 1.0 ships" as the original review assumed. This pass + kept `Compono.MSTest`/`Compono.NUnit` on `--prerelease`, since + neither had a stable release (verified live against nuget.org). + 2. A Codex review of PR #131 correctly flagged a bug this introduced: + `Compono.MSTest`/`Compono.NUnit` each pin their `Compono` dependency + to their own **exact** package version (`[%(ProjectVersion)]` bracket + syntax in each `.csproj`'s `PinProjectReferenceVersionsExact` + target) — verified directly against the packed `.nuspec` on + nuget.org: `Compono.MSTest`/`Compono.NUnit` `0.10.0-preview.101` both + declare ``. + The docs' shared "`dotnet add package Compono`" line (resolving + stable `0.9.0`) followed by `Compono.MSTest`/`Compono.NUnit + --prerelease` (resolving `0.10.0-preview.101`) taught an invalid + pairing — NuGet cannot satisfy an exact `[0.10.0-preview.101]` + dependency with an already-resolved `0.9.0` core. + 3. Before that fix shipped, the product owner directed a different, + simpler resolution: `Compono.MSTest`/`Compono.NUnit` are about to go + stable together with every other package immediately after this PR + merges, so the docs now install all six packages + (`Compono`/`XunitV3`/`TUnit`/`MSTest`/`NUnit` plus the four + independent add-ons) the same way, with no `--prerelease` anywhere + and no stable/preview distinction — correct once that release lands, + and this record notes plainly that the docs are written slightly + ahead of the actual publish rather than pretending otherwise. +- **[F4]** — the ADR-0031 TFM discrepancy. **Investigated and found to be a + false positive**, not a real drift: ADR-0031's own **Amendment 3** + (2026-08-10) already records, in the same file, that ADR-0038 + superseded the original two-TFM framing and that + `net8.0;net9.0;net10.0;net11.0` are all actively-tracked TFMs. The + original review's finding came from reading only the ADR's original + "Decision Outcome" prose (2026-08-04) without reading its own later + amendments in the same document. No ADR/documentation change was + needed — the decision record was already accurate. Nothing in + current-facing docs (README, package guides) states a TFM count that + would need correcting either. +- **[F5]** — dogfood Docker prerequisite. Resolved: `scripts/dogfood-validate.sh`'s + usage text now states the prerequisite and its failure mode explicitly. +- The README's "Status" section ("APIs are experimental until the first + public preview") was also already stale *today* — Compono has stable + `0.9.0` packages, not merely "preview." Corrected to state plainly: + publicly released, stable `0.x` packages exist, still pre-`1.0` so the + API can still change. +- **New in this round: Compono skill eval/workspace structure** — brought + into alignment with the current [Agent Skills eval-workspace + convention](https://agentskills.io/skill-creation/evaluating-skills). + Authored `evals.json` moved from the sibling `skills/compono-evals/` + directory (a 2026-08-07 fix, at the time correct for the ecosystem + convention as it existed then — see PLAN-0035's own "Real defect" + note) into `skills/compono/evals/evals.json`, matching the doc's + canonical `/evals/evals.json` layout; the generated-workspace + half (`benchmarks/`, all historical, preserved as-is) moved to a new + sibling `skills/compono-workspace/`, matching the doc's + `-workspace/` sibling convention, with a new + `skills/compono-workspace/README.md` documenting the current + `iteration-N/`/`with_skill`+`old_skill`/`timing.json`/`grading.json`/ + `benchmark.json`/`feedback.json` shape for future runs. **Trade-off + reintroduced, flagged rather than silently accepted:** this move puts + `evals.json` (51KB, 46 scenarios) back inside the directory `npx skills + add` copies verbatim to every consumer installing the `compono` skill — + exactly the bloat PLAN-0035's 2026-08-07 move was built to avoid. The + current agentskills.io specification and eval-workflow doc were both + checked directly and neither documents any installer-side exclusion + mechanism for `evals/`; the doc's own canonical example ships + `evals/evals.json` (and any `evals/files/`) as part of the installed + skill. This is a genuine, upstream-level tension between "evals belong + in the skill directory" (current convention) and "evals shouldn't ship + to every consumer" (this repo's original, still-valid concern) — not + something this pass invented or silently resolved. Implemented as + directed because the instruction was explicit and precisely specified, + not exploratory; flagged here as a decision the product owner may want + to revisit if consumer-side skill-install size becomes a real + complaint (e.g. by asking upstream for a `.skillsignore`-equivalent, or + reconsidering the trade-off). + +**Explicitly rejected as unnecessary pre-1.0 gates (5):** + +- **[F2]** (additional packed-consumer smoke projects for the remaining 6 + packages) — the product owner determined the original Logging finding + was specifically a ProjectReference-vs-packed-MSBuild-assets + development concern (already correctly scoped and handled — see + [PLAN-0061](../plans/0061-pre-1-0-cleanup-and-consolidation.md) Phase + 2 and the packed asset's own inline documentation), and that the + packed nupkg's assets for those 6 packages are already inspected by + `inspect-packed-nupkgs.sh`. No new concrete evidence of an actual + uncovered packaging failure mode was presented, so six more + packed-consumer scenarios were not added. +- **[F6]** (20 xUnit-analyzer warnings) — no shipped-product correctness + impact; left for separate follow-up. +- Additional release-drafter machinery around `breaking-change → major` + — that mapping (ADR-0031 Amendment 5) is intentional, already-decided + SemVer behavior for leaving `0.x`; publishing itself stays + human-controlled regardless. No new gate added. +- Re-litigating disposal/lifetime ownership, async composition, or + framework-binder consolidation — the adversarial review found no + reason for any of these to block 1.0; not reopened. +- Additional AOT projects without a contradicted support claim (e.g. for + `Compono.Bogus`/`Compono.DependencyInjection`, which document no AOT + claim either way) — not added; nothing regresses by adding this later. + +## Final disposition + +**READY FOR 1.0**, pending only the release-execution act itself. The +product owner has stated intent to cut `1.0.0` for every package +immediately once PR #131 merges — F3's final documentation state (§ +above) is written for that near-term stable world, not the transient +state that existed while `Compono.MSTest`/`Compono.NUnit` were still +preview-only. All five accepted pre-1.0/hardening items above are +resolved and verified, including the required-status-check change now +confirmed live on `main`. No item originally listed as a Release Task, +and later rejected, should be read as a mandatory gate — each rejection +above is a deliberate product-owner decision, not an oversight. + +## Links + +- Original adversarial review — conducted in-session; no separate + research record existed for it prior to this one, which supersedes it + as the authoritative write-up of both the original findings and their + disposition. +- [PLAN-0061](../plans/0061-pre-1-0-cleanup-and-consolidation.md), + [PLAN-0062](../plans/0062-package-validation-gap-fixes.md) — the + pre-1.0 work this review verified. +- [ADR-0031](../adr/0031-public-preview-release-and-versioning-policy.md) — + release/versioning policy, including Amendment 5 (`0.x` graduation) + referenced above. +- [ADR-0038](../adr/0038-net8-net9-explicit-multi-target.md) — the + four-TFM decision ADR-0031 Amendment 3 already reconciles against. +- `LayeredCraft/.github` PR #5 — safe-settings repo-managed + required-checks opt-out for `compono`. diff --git a/docs/troubleshooting/faq.md b/docs/troubleshooting/faq.md index 35d20a76..dcfb5837 100644 --- a/docs/troubleshooting/faq.md +++ b/docs/troubleshooting/faq.md @@ -56,12 +56,17 @@ instance every call) rather than a fresh one. ## Is there a stable `1.0` release yet? -Not yet — every published version during public preview is a -`0.x.y-preview.N` prerelease (`--prerelease` required to install). See -[MVP Non-goals](../mvp.md#mvp-non-goals) and +Not yet, but every package already has a stable release — a plain +`dotnet add package` picks it up with no extra flag. See +[Installation](../getting-started/installation.md) for exact commands, +[MVP Non-goals](../mvp.md#mvp-non-goals), and [ADR-0031](../adr/0031-public-preview-release-and-versioning-policy.md) -for the compatibility policy this implies: a breaking change bumps the -minor version (`0.X+1.0`), not a major version, until `1.0`. +for the compatibility policy this implies. Note ADR-0031's own Amendment 5: +a breaking change now bumps the major version (the ordinary SemVer +`breaking-change → major` mapping), not the minor version — the ADR's +original "bumps the minor version until `1.0`" override was a deliberate, +temporary `0.x`-era guard that's since been lifted now that Compono is +ready to leave the `0.x` line. ## Next diff --git a/scripts/dogfood-validate.sh b/scripts/dogfood-validate.sh index 49ef61b6..f5a9ace3 100755 --- a/scripts/dogfood-validate.sh +++ b/scripts/dogfood-validate.sh @@ -85,6 +85,13 @@ Exit code is 0 only if packing, restore, version verification (every requested p the exact freshly-packed version - not a stale cache hit, and not a mix of freshly-packed and previously published versions), and the consumer's full test suite all succeed. The consumer repo's git working tree is left exactly as it was found, regardless of outcome. + +Prerequisite: the default consumer (trivia-platform) includes Testcontainers-backed repository +tests that require a running Docker daemon. Without one, those tests fail with +DockerUnavailableException and this script exits non-zero even when every Compono package under +test is fine - that's an environment gap, not a Compono regression. Start Docker (or point +--consumer-repo/DOGFOOD_CONSUMER_REPO at a consumer that doesn't need it) before relying on this +script's exit code as a pass/fail signal. EOF } diff --git a/skills/compono-workspace/README.md b/skills/compono-workspace/README.md new file mode 100644 index 00000000..f4d9d81e --- /dev/null +++ b/skills/compono-workspace/README.md @@ -0,0 +1,66 @@ +# `compono` skill eval workspace + +Generated eval-execution evidence for the `compono` agent skill +(`skills/compono/`), kept alongside the skill directory per the +[Agent Skills eval-workspace convention](https://agentskills.io/skill-creation/evaluating-skills) — +authored eval definitions live in `skills/compono/evals/evals.json`; +everything here is generated output, never authored skill content. + +## Structure + +``` +compono-workspace/ +├── skill-snapshot/ # cp -r of skills/compono/ taken before an edit, when +│ # comparing an iteration against the previous skill +│ # revision rather than against no skill at all +├── iteration-N/ +│ ├── eval-/ +│ │ ├── with_skill/ +│ │ │ ├── outputs/ # files the run produced +│ │ │ ├── timing.json # { "total_tokens": ..., "duration_ms": ... } +│ │ │ └── grading.json # assertion_results + summary, per evals.json's +│ │ │ # "assertions"/"expectations" for that case +│ │ └── old_skill/ # or without_skill/ - see "Baseline choice" below +│ │ ├── outputs/ +│ │ ├── timing.json +│ │ └── grading.json +│ ├── benchmark.json # aggregated pass-rate/time/token stats for the iteration +│ └── feedback.json # { "": "", ... } +└── benchmarks/ # pre-convention historical runs (2026-08-07 through + # 2026-09-03) - kept as-is; each is its own dated + # README/benchmark.md, not restructured into the + # iteration-N/ shape above +``` + +## Baseline choice + +`compono` is an established skill under active iteration, not a new skill +being evaluated from zero — per the convention above, a revision compares +against a `skill-snapshot/` of the previous version (`old_skill/`), not +against no skill (`without_skill/`), unless the run is specifically +measuring whether having the skill at all is worth it (in which case use +`without_skill/` and say so explicitly in that iteration's directory). + +## Clean-context execution + +Each with-skill/baseline run in an iteration starts from a fresh +subagent (the `Agent` tool's `general-purpose` type, or a fresh Claude +Code session if run outside this repo) — never a continuation of the +skill-development conversation. This is what makes a run's output +attributable to `SKILL.md` itself rather than to prior conversational +state. Each run is given: the skill path (or none, or the snapshot path), +the eval's `prompt`, its `files` (if any), and a distinct output directory +so concurrent with-skill/baseline runs can never collide. + +## Grading + +Grade each case's `assertions`/`expectations` (from `evals/evals.json`) +against that run's `outputs/`, with concrete evidence per PASS/FAIL — +per this repo's own established practice (see `benchmarks/`), grading is +currently done by direct reasoning against the skill's actual reference +content, not an automated content-grading harness; +`.agents/skills/skill-creator/scripts/run_eval.py` is a *trigger*-eval +runner (does the skill activate at all), not a content grader. Prefer a +deterministic script wherever an assertion is mechanically checkable +(valid syntax, a specific API name present/absent); reserve reasoning-based +grading for assertions that genuinely require it. diff --git a/skills/compono-evals/benchmarks/2026-08-07/README.md b/skills/compono-workspace/benchmarks/2026-08-07/README.md similarity index 96% rename from skills/compono-evals/benchmarks/2026-08-07/README.md rename to skills/compono-workspace/benchmarks/2026-08-07/README.md index 6426b3d9..31a1147a 100644 --- a/skills/compono-evals/benchmarks/2026-08-07/README.md +++ b/skills/compono-workspace/benchmarks/2026-08-07/README.md @@ -1,7 +1,7 @@ # Benchmark run — 2026-08-07 A `/skill-creator`-*style* eval run against all 18 scenarios in -`../../evals.json` (superseding the 6-scenario manual spot-check recorded +`../../../compono/evals/evals.json` (superseding the 6-scenario manual spot-check recorded earlier in PLAN-0035) — not `/skill-creator`'s full documented workflow; see "Known limitations" below for exactly where it's lighter-weight. With-skill and baseline (`without_skill`, no access to the skill's @@ -89,7 +89,7 @@ follow-up for whoever next revises `evals.json`, per this repo's ## Superseded eval manifest -`../../evals.json` now contains 26 scenarios. This benchmark remains an +`../../../compono/evals/evals.json` now contains 26 scenarios. This benchmark remains an accurate historical record for scenarios 1-18 only; it does **not** measure the later additions for parameterized profiles, NUnit, TUnit, generated test doubles, dependency injection, custom providers, `CreateMany`, or diff --git a/skills/compono-evals/benchmarks/2026-08-07/benchmark.json b/skills/compono-workspace/benchmarks/2026-08-07/benchmark.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/benchmark.json rename to skills/compono-workspace/benchmarks/2026-08-07/benchmark.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/benchmark.md b/skills/compono-workspace/benchmarks/2026-08-07/benchmark.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/benchmark.md rename to skills/compono-workspace/benchmarks/2026-08-07/benchmark.md diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-1-use-compono-to-create-the-request-model-/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-10-my-integration-test-spins-up-a-webapplic/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-11-why-does-my-test-throw-nullreferenceexce/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-12-i-want-to-register-iclock-twice-in-my-pr/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-13-should-i-mark-my-entire-domain-model-wit/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-14-write-a-generic-fibonacci-function-in-c-/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-15-this-project-already-uses-compono-and-co/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-16-compono-can-t-compose-httpclient-in-my-t/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-17-compono-keeps-throwing-compositionexcept/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-18-give-me-a-realistic-looking-customer-wit/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-2-why-is-compono-failing-to-compose-this-t/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-3-convert-this-xunit-test-that-manually-bu/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-4-i-want-the-same-nsubstitute-dependency-r/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-5-use-bogus-for-the-email-address-but-let-/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-6-create-a-theory-using-compono-with-a-fix/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-7-why-did-adding-composable-not-fix-this-i/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-8-set-up-a-test-project-from-scratch-using/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/with_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/with_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/with_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/with_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/without_skill.json b/skills/compono-workspace/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/without_skill.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/without_skill.json rename to skills/compono-workspace/benchmarks/2026-08-07/grading/eval-9-this-nsubstitute-test-doubles-a-service-/without_skill.json diff --git a/skills/compono-evals/benchmarks/2026-08-25/README.md b/skills/compono-workspace/benchmarks/2026-08-25/README.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-25/README.md rename to skills/compono-workspace/benchmarks/2026-08-25/README.md diff --git a/skills/compono-evals/benchmarks/2026-08-25/benchmark.md b/skills/compono-workspace/benchmarks/2026-08-25/benchmark.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-25/benchmark.md rename to skills/compono-workspace/benchmarks/2026-08-25/benchmark.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/benchmark.json b/skills/compono-workspace/benchmarks/2026-08-27/benchmark.json similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/benchmark.json rename to skills/compono-workspace/benchmarks/2026-08-27/benchmark.json diff --git a/skills/compono-evals/benchmarks/2026-08-27/benchmark.md b/skills/compono-workspace/benchmarks/2026-08-27/benchmark.md similarity index 98% rename from skills/compono-evals/benchmarks/2026-08-27/benchmark.md rename to skills/compono-workspace/benchmarks/2026-08-27/benchmark.md index 4608ee10..16b87c1e 100644 --- a/skills/compono-evals/benchmarks/2026-08-27/benchmark.md +++ b/skills/compono-workspace/benchmarks/2026-08-27/benchmark.md @@ -4,7 +4,7 @@ Focused regression run for the AWS Secrets Manager Provider TestDoubles matching ## Scope -New evals added in `skills/compono-evals/evals.json`: +New evals added in `skills/compono/evals/evals.json`: - 28 — `IConfigurationBuilder` argument-filtered verification - 29 — `IAmazonSecretsManager` argument-matched configuration diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/eval-28-with-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-28-with-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/eval-28-with-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-28-with-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/eval-29-with-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-29-with-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/eval-29-with-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-29-with-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/eval-30-with-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-30-with-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/eval-30-with-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-30-with-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/eval-31-with-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-31-with-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/eval-31-with-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/eval-31-with-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-28-old-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-28-old-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-28-old-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-28-old-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-29-old-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-29-old-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-29-old-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-29-old-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-30-old-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-30-old-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-30-old-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-30-old-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-31-old-skill.md b/skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-31-old-skill.md similarity index 100% rename from skills/compono-evals/benchmarks/2026-08-27/outputs/old_skill/eval-31-old-skill.md rename to skills/compono-workspace/benchmarks/2026-08-27/outputs/old_skill/eval-31-old-skill.md diff --git a/skills/compono-evals/benchmarks/2026-08-28/README.md b/skills/compono-workspace/benchmarks/2026-08-28/README.md similarity index 96% rename from skills/compono-evals/benchmarks/2026-08-28/README.md rename to skills/compono-workspace/benchmarks/2026-08-28/README.md index 77a08db8..e6a60c6f 100644 --- a/skills/compono-evals/benchmarks/2026-08-28/README.md +++ b/skills/compono-workspace/benchmarks/2026-08-28/README.md @@ -1,7 +1,7 @@ # Benchmark run — 2026-08-28 Scoped, single-eval run for the new `Share()` eval added by PLAN-0056 -Task 6 (id 34 in `../../evals.json`) — not a full re-run of all 34 +Task 6 (id 34 in `../../../compono/evals/evals.json`) — not a full re-run of all 34 scenarios. With-skill and without-skill (`without_skill`, told not to read `skills/compono/`) subagents ran independently against eval 34's prompt, graded manually against its `expectations` (5 assertions). diff --git a/skills/compono-evals/benchmarks/2026-09-02/README.md b/skills/compono-workspace/benchmarks/2026-09-02/README.md similarity index 97% rename from skills/compono-evals/benchmarks/2026-09-02/README.md rename to skills/compono-workspace/benchmarks/2026-09-02/README.md index 1beeda52..e11b0565 100644 --- a/skills/compono-evals/benchmarks/2026-09-02/README.md +++ b/skills/compono-workspace/benchmarks/2026-09-02/README.md @@ -1,7 +1,7 @@ # Benchmark run — 2026-09-02 Scoped, two-eval run for the new `Compono.MSTest` evals added by -PLAN-0057 task group 14 (ids 40-41 in `../../evals.json`) — not a full +PLAN-0057 task group 14 (ids 40-41 in `../../../compono/evals/evals.json`) — not a full re-run of all 41 scenarios. Graded by the implementing session directly (same limitation as the 2026-08-28 run), reasoning through a with-skill answer (grounded in `skills/compono/references/mstest.md`'s actual diff --git a/skills/compono-evals/benchmarks/2026-09-03/README.md b/skills/compono-workspace/benchmarks/2026-09-03/README.md similarity index 98% rename from skills/compono-evals/benchmarks/2026-09-03/README.md rename to skills/compono-workspace/benchmarks/2026-09-03/README.md index 9bf77210..93a86cf5 100644 --- a/skills/compono-evals/benchmarks/2026-09-03/README.md +++ b/skills/compono-workspace/benchmarks/2026-09-03/README.md @@ -1,7 +1,7 @@ # Benchmark run — 2026-09-03 Scoped, five-eval run for the new `Compono.NUnit` evals added by -PLAN-0059 (ids 42-46 in `../../evals.json`) — not a full re-run of all 46 +PLAN-0059 (ids 42-46 in `../../../compono/evals/evals.json`) — not a full re-run of all 46 scenarios. Graded by the implementing session directly (same limitation as the 2026-08-28/2026-09-02 runs — no automated content-grading harness exists for this repo's `evals.json` prompt/expected_output format; diff --git a/skills/compono-workspace/iteration-6/benchmark.json b/skills/compono-workspace/iteration-6/benchmark.json new file mode 100644 index 00000000..bdde64cc --- /dev/null +++ b/skills/compono-workspace/iteration-6/benchmark.json @@ -0,0 +1,20 @@ +{ + "purpose": "Structural validation of the skills/compono/evals/ + skills/compono-workspace/ relocation (RESEARCH-0022 follow-up) - proves eval discovery, clean-context execution, output separation, timing capture, and grading still work after the move. Single eval (id 1), not a full 46-scenario re-run; not intended to drive a SKILL.md change.", + "run_summary": { + "with_skill": { + "pass_rate": { "mean": 1.0 }, + "tokens": { "mean": 53722 }, + "duration_ms": { "mean": 54234 } + }, + "without_skill": { + "pass_rate": { "mean": 0.667 }, + "tokens": { "mean": 31624 }, + "duration_ms": { "mean": 23619 } + }, + "delta": { + "pass_rate": 0.333, + "tokens": 22098, + "duration_ms": 30615 + } + } +} diff --git a/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/grading.json b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/grading.json new file mode 100644 index 00000000..38c28649 --- /dev/null +++ b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/grading.json @@ -0,0 +1,25 @@ +{ + "assertion_results": [ + { + "text": "Uses only real Compono APIs (Composer.Create(...) to build the composer, composer.Create()/CreateMany() as instance methods, [Compose], etc.), nothing invented, and distinguishes programmatic composition from a Compose row", + "passed": true, + "evidence": "Option A uses [Theory][Compose] as a real Compono.XunitV3 attribute; Option B uses `Composer.Create()` (static factory) then the instance method `Composer.Create()` - no invented API, and the two options are explicitly labeled 'inside an xUnit v3 theory' vs 'inside the test body (arrange step, not a theory parameter)', a clear programmatic-vs-Compose-row distinction." + }, + { + "text": "Does not add [Composable] unless there is a genuine discovery-gap reason", + "passed": true, + "evidence": "Explicit 'Why not other approaches' section: 'No [Composable] on CreateOrderRequest... the type is composed directly at a visible call site... adding the attribute would be speculative, not a fix for anything broken.'" + }, + { + "text": "Does not introduce Activator.CreateInstance or any reflection-based fallback", + "passed": true, + "evidence": "Explicit: 'No Activator.CreateInstance or other reflection... Compono is source-generated by design (ADR-0001) and has no reflection fallback.'" + } + ], + "summary": { + "passed": 3, + "failed": 0, + "total": 3, + "pass_rate": 1.0 + } +} diff --git a/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/timing.json b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/timing.json new file mode 100644 index 00000000..28117b47 --- /dev/null +++ b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/with_skill/timing.json @@ -0,0 +1,4 @@ +{ + "total_tokens": 53722, + "duration_ms": 54234 +} diff --git a/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/grading.json b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/grading.json new file mode 100644 index 00000000..5a6b3392 --- /dev/null +++ b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/grading.json @@ -0,0 +1,28 @@ +{ + "assertion_results": [ + { + "text": "Uses only real Compono APIs (Composer.Create(...) to build the composer, composer.Create()/CreateMany() as instance methods, [Compose], etc.), nothing invented, and distinguishes programmatic composition from a Compose row", + "passed": false, + "evidence": "Invents `[ComposableTest]` (not a real Compono.XunitV3 attribute - the real attribute is `[Compose]` on a `[Theory]`) and `new Composer()` (Compono's Composer is constructed via the static `Composer.Create()` factory, not a public constructor)." + }, + { + "text": "Does not add [Composable] unless there is a genuine discovery-gap reason", + "passed": true, + "evidence": "Never mentions or adds [Composable] - a baseline pass by omission rather than by correct reasoning about the attribute's purpose." + }, + { + "text": "Does not introduce Activator.CreateInstance or any reflection-based fallback", + "passed": true, + "evidence": "No reflection introduced - a baseline pass by omission rather than by correct reasoning about Compono's source-generation design." + } + ], + "summary": { + "passed": 2, + "failed": 1, + "total": 3, + "pass_rate": 0.667 + }, + "additional_observations": [ + "Recommends post-hoc mutation ('request.Quantity = 5;' after Create()) as the way to pin specific values - directly contradicts the with-skill answer's explicit member-rule/CompositionBuilder guidance and the general Compono philosophy against composing then mutating. Not covered by an eval-1 assertion, but a real quality gap flagged as human-review feedback (see feedback.json)." + ] +} diff --git a/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/timing.json b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/timing.json new file mode 100644 index 00000000..6d462128 --- /dev/null +++ b/skills/compono-workspace/iteration-6/eval-1-use-compono-to-create-the-request-model/without_skill/timing.json @@ -0,0 +1,4 @@ +{ + "total_tokens": 31624, + "duration_ms": 23619 +} diff --git a/skills/compono-workspace/iteration-6/feedback.json b/skills/compono-workspace/iteration-6/feedback.json new file mode 100644 index 00000000..0ca2871f --- /dev/null +++ b/skills/compono-workspace/iteration-6/feedback.json @@ -0,0 +1,3 @@ +{ + "eval-1-use-compono-to-create-the-request-model": "With-skill answer is correct and matches the eval's assertions. No SKILL.md change proposed from this single-eval structural-validation run - not a full iteration." +} diff --git a/skills/compono-evals/evals.json b/skills/compono/evals/evals.json similarity index 100% rename from skills/compono-evals/evals.json rename to skills/compono/evals/evals.json