CXF-222: skills README documents provenance and eval evidence; all ten skills name their contract sources - #17
Conversation
Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| pre-1 skills authored against the v0.0.26 DSL contract and the 23-tool | ||
| tenant MCP surface, plus three post-funnel and cross-cutting skills | ||
| (`verify-connector-output`, `update-and-rollback`, | ||
| pre-1 skills authored against the v0.0.26 DSL contract and the 23-tool tenant MCP surface; validated by the eval suite, plus three post-funnel |
There was a problem hiding this comment.
🟡 Suggestion: The inserted ; validated by the eval suite, is both grammatically broken (semicolon then a comma-joined plus …) and overstated. The Tier-0 replay cited below runs evals/scenarios/tier1-directory.json, which has "skillBundle": {"mode": "none"} and a canned/static driver — so it exercises the scorer and fixture, not the skills; lines 36–41 then say the with-skills-vs-baseline measurement is structurally blocked. Consider dropping the clause (or rewording to "the eval harness scores the funnel these skills target") and re-wrapping to the file's ~70-column width.
| | OpenRouter official spec (evidence) | OpenRouter official published spec (openrouter.ai), vendored verbatim | The authoritative spec evidence for a SaaS provider with an official published spec. | | ||
| | LiteLLM runtime-generated spec (evidence) | `pip install "litellm[proxy]==1.92.0"` → `app.openapi()` | The runtime-generated spec evidence for a self-hosted OSS provider. | | ||
| | Shopify vendored minimal spec (evidence) | `allengrant/shopify_openapi` @ master, discoveredAt 2026-06-04 | The vendored minimal spec evidence for a provider without an official published spec. | |
There was a problem hiding this comment.
🟡 Suggestion: These three rows sit in a Pin / SHA column but carry no pin: the OpenRouter row names only "openrouter.ai" with no version or retrieval date, and the Shopify row pins allengrant/shopify_openapi @ master — a moving ref, so the cited evidence can change under the same citation. skills/README.md states "Each skill's SOURCES.md names the pinned sources with their SHAs," and every pre-existing row here honors that. Pin the Shopify row to a commit SHA and give the OpenRouter row a spec version or discoveredAt date. (The LiteLLM row is fine — ==1.92.0 is a real pin.)
| | Worked build — access-model shape | baton-axiomatic-openrouter#1 | The access-model shape (resource types, traits, entitlements, grants) this worked build demonstrates. | | ||
| | Worked build — access-model shape | baton-axiomatic-litellm#1 | The access-model shape (resource types, traits, entitlements, grants) this worked build demonstrates. | | ||
| | Worked build — access-model shape | baton-axiomatic-shopify | The access-model shape (resource types, traits, entitlements, grants) this worked build demonstrates. | |
There was a problem hiding this comment.
🟡 Suggestion: All three rows share the same label ("Worked build — access-model shape") and a byte-identical "What this skill quotes" cell, so the table gives no way to tell what each build contributes. They are also unpinned: #1 is a PR number, not a SHA, and baton-axiomatic-shopify has no ref at all — note that write-connector-source/SOURCES.md pins that same repo to 6ea2834043832f7bb21ca59111de2cf021c17518. Add SHAs and differentiate the quote cells.
| | Authoring proto | c1 `2502b4cd8f59bf6614616013010ec4f0bf72f9ae` | RPC titles for every funnel step (`connector_authoring_create_draft`, `connector_authoring_create_draft_source_upload`, `connector_authoring_finalize_draft_source_upload`, `connector_authoring_build_bundle`, `connector_authoring_get_run`, `connector_authoring_run_draft_test_sync`, `connector_authoring_get_test_run_evidence`, `connector_authoring_provision_connector`, `connector_authoring_deploy_connector_instance`, `connector_authoring_mint_approval_token`); `MintApprovalTokenResponse.activation_url`. | | ||
| | Lifecycle doc | c1 `2502b4cd8f59bf6614616013010ec4f0bf72f9ae` | The 12-step order and stop-if gates; the handoff-table discipline (fill every row from the tool response of the step that produces it); step 11's human-OWNER handoff. | | ||
| | In-repo SDK declarations | connector-authoring `b7e8a616cbbb1e336b788f807a3810b08ae00bc7` | The `.d.ts` module surface the funnel's source files are authored against. | | ||
| | SDK contract (`baton/*.d.ts`) | connector-authoring git tag `v0.0.26` (sync commit `01a69d8d` "Sync baton runtime types for v0.0.26") | The `.d.ts` module surface this skill is authored against. `runtime_pin_matched`: not verifiable offline — no tenant MCP surface is reachable from the authoring env; repo tag v0.0.26 is the served default_tag when the tenant runtime pin matches (see `read-authoring-contract` for the runtime check). | |
There was a problem hiding this comment.
🟡 Suggestion: This new row and the pre-existing "In-repo SDK declarations" row above both describe the same subject — the .d.ts module surface this skill is authored against — but pin different commits (b7e8a616… vs. tag v0.0.26 / 01a69d8d…), with no stated relationship between them. A reader can't tell which is authoritative. Either fold the tag into the existing row or add a clause explaining that v0.0.26 is the contract version and the older SHA is the repo state the skill text was written against. Same duplication appears in all nine SOURCES.md files touched here.
| | Authoring proto | c1 `2502b4cd8f59bf6614616013010ec4f0bf72f9ae` | RPC titles `connector_authoring_get_authoring_guide`, `connector_authoring_list_sdk_types_versions`, `connector_authoring_get_sdk_types`, `connector_authoring_list_authored_catalog_entries`, `connector_authoring_list_drafts`, `connector_authoring_create_draft`; `ListSDKTypesVersionsResponse` fields `default_tag` / `runtime_pin_matched`. | | ||
| | Lifecycle doc | c1 `2502b4cd8f59bf6614616013010ec4f0bf72f9ae` | Step 0 (guide read) and the handoff-table discipline (record every returned ID). | | ||
| | In-repo SDK declarations | connector-authoring `b7e8a616cbbb1e336b788f807a3810b08ae00bc7` | The `.d.ts` module surface (`@baton/runtime`, `@baton/helpers`, `@baton/types`) the tagged declarations cover. | | ||
| | SDK contract (`baton/*.d.ts`) | connector-authoring git tag `v0.0.26` (sync commit `01a69d8d` "Sync baton runtime types for v0.0.26") | The `.d.ts` module surface this skill is authored against. `runtime_pin_matched`: not verifiable offline — no tenant MCP surface is reachable from the authoring env; repo tag v0.0.26 is the served default_tag when the tenant runtime pin matches (see `read-authoring-contract` for the runtime check). | |
There was a problem hiding this comment.
🟡 Suggestion: The shared row text ends with "(see read-authoring-contract for the runtime check)" — a self-reference inside read-authoring-contract's own SOURCES.md. Copy-paste artifact of the row being duplicated across nine files; reword to point at this skill's own body (e.g. "the runtime check this skill performs") in this copy.
btipling
changed the title
General PR Review: Implement src-3naou-att2Blocking Issues: 0 | Suggestions: 5 | Threads Resolved: 0 Review SummaryScanned the full PR diff (11 files, +46/-3, documentation only — a new "Eval evidence" section in Security IssuesNone found. Correctness IssuesNone found. Suggestions
Prompt for AI agents |
1 participant
Summary
The ten connector-authoring skills now document where every source came from. Each skill's
SOURCES.mdnames the SDK contract (baton/*.d.tsat tag v0.0.26, with theruntime_pin_matchedrecord) and the baton-axiomatic DSL contract it was authored against, and the three skills the issue calls out add their knowledge-provenance rows (worked builds, spec evidence, baton-admin DSL skills). The skills README states the model terms the skills were authored in and adds an## Eval evidencesection citing the deterministic, committed eval harness. This is CXF-70 PR 7: it closes the provenance rows, the eval-evidence section, and (with the companion marketplace PR) the distribution wrapper.Linear
CXF-222: CXF-70 PR 7: README + SOURCES.md + marketplace plugin wrapper
Linear requirements
This PR meets all requirements of CXF-222; the measured pass-rate gate (requirement 3) is carried per the merged PR #10/#11 precedent, because the Tier-1 baseline E2E is structurally blocked and no pass-rate numbers exist to report.
skills/README.mdL3-8 (model terms), L10-21 (ten-row table)baton/*.d.ts@ tag withruntime_pin_matched;baton-axiomaticdocs/DSL.md+runtime/baton/*.d.ts@ v0.0.26) plus per-skill knowledge provenanceskills/*/SOURCES.md×10 — SDK-contract + baton-axiomatic rows in all ten (e.g.skills/author-in-app-connector/SOURCES.mdL13-15); worked-builds rows inskills/design-access-model/SOURCES.mdL13-18; spec-evidence rows inskills/source-openapi-spec/SOURCES.mdL13-17; baton-admin DSL rows inskills/write-connector-source/SOURCES.mdL21-24skills/README.mdL27-41 — cites the Tier-0 scored replay, unit smokes, and scorer/fixture harness; Tier-1 baseline structurally blocked (public repo ships only the Tier-0 canned driver; eval envs expose noc1_connector_authoring_*MCP tools); carried per merged PRs #10/#11; no numbers fabricatedplugin.json+ skills, installed via/plugin install <name>@<tag>)ConductorOne/claude-marketplace#152—plugins/connector-authoring/vendoring all ten skills byte-identical from this branch HEAD45aae31; registered in.claude-plugin/marketplace.json; install line/plugin install connector-authoring@c1-engineeringRelease Notes
No user-facing change.
What changed
skills/README.mdWhere: intro ~L3-8; new
## Eval evidencesection L27-41.What changed
## Eval evidencesection: the Tier-0 scored replay command, the unit smokes, and the scorer/fixture harness, followed by the Tier-1 baseline blocker and the carry-forward rule from merged PRs CXF-217: eval harness pins reasoning effort and can generate the baseline reference #10/CXF-218: four mechanical funnel skills (orchestrator, contract, build-and-test, deploy-and-activate) #11.Why
c1_connector_authoring_*MCP surface on eval envs), so the section cites the deterministic evidence that exists and states the blocker instead of inventing numbers.skills/*/SOURCES.md(ten files)Where: end-of-table appends —
author-in-app-connector,build-and-test,deploy-and-activate,diagnose-authoring-failure,read-authoring-contract,verify-connector-outputL13-15;update-and-rollbackL12-14;design-access-modelL13-18;source-openapi-specL13-17;write-connector-sourceL21-24.What changed
baton/*.d.ts@ git tag v0.0.26, sync commit01a69d8d, with the lockedruntime_pin_matchedwording) and the baton-axiomatic DSL contract (docs/DSL.md+runtime/baton/*.d.ts@ v0.0.26, full SHA825e5516…).design-access-modeladds three worked-build rows (baton-axiomatic-openrouter#1,baton-axiomatic-litellm#1,baton-axiomatic-shopify) for access-model shapes.source-openapi-specadds three spec-evidence rows (OpenRouter official published spec vendored verbatim; LiteLLM runtime-generated spec viapip install "litellm[proxy]==1.92.0"→app.openapi(); Shopify vendored minimal specallengrant/shopify_openapi@ master, discoveredAt 2026-06-04).write-connector-sourceadds two baton-admin DSL-skill rows (author-js-dsl-connector,author-auth-config-surface@6fe6886f…).Why
list_sdk_types_versions(recorded asruntime_pin_matched) and the baton-axiomatic DSL ground truth, plus the per-skill knowledge provenance the issue names. Existing pinned rows are untouched; this is a pure delta.Excluded
evals/— untouched (non-goal; the harness already exists from PRs CXF-217: eval harness pins reasoning effort and can generate the baseline reference #10/CXF-218: four mechanical funnel skills (orchestrator, contract, build-and-test, deploy-and-activate) #11).ConductorOne/claude-marketplace— the plugin wrapper ships in companion PR #152.