diff --git a/AGENTS.md b/AGENTS.md index 164799a..994e30f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -211,7 +211,7 @@ Bare `yolo` or `auto` outside an explicit commit request does not invoke `git-vi ### PR Skill Routing -When the user asks to create, open, make, or refresh a GitHub pull request, invoke `git-remote-pr` before remote PR operations. Normal requests inspect the complete committed `base...head` changeset, prepare a title/body and exact mutation preview, then require explicit approval before `git push` or PR writes. `yolo`/`auto` attached to that same explicit PR request authorizes those narrow writes after the preview is shown; bare `yolo`/`auto` does not invoke the skill. Existing PR descriptions are regenerated from the complete current changeset. Dirty worktrees are not auto-committed, and approval never authorizes force pushes or history rewriting. Commit requests remain with `git-visual-commits`; release-note requests remain with `git-remote-release`. `/git-remote-pr` can force deterministic selection where supported. +When the user asks to create, open, make, or refresh a GitHub pull request, invoke `git-remote-pr` before remote PR operations. Normal requests inspect the complete committed `base...head` changeset, prepare a title/body and exact mutation preview, then require explicit approval before `git push` or PR writes. `yolo`/`auto` attached to that same explicit PR request still shows the preview but skips the approval wait: do not ask for or wait on `approve APR-...`; continue only with the just-presented plan and APR ID. Bare `yolo`/`auto` does not invoke the skill. Existing PR descriptions are regenerated from the complete current changeset. Dirty worktrees are not auto-committed, and approval never authorizes force pushes or history rewriting. Commit requests remain with `git-visual-commits`; release-note requests remain with `git-remote-release`. `/git-remote-pr` can force deterministic selection where supported. ## Skill Creation diff --git a/CHANGELOG.md b/CHANGELOG.md index ef200f1..f375c71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,23 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.11.1] - 2026-09-29 + +This is a patch release delivering validation hardening for `git-remote-release` with stricter em-dash and alert-block enforcement, comprehensive test coverage for format compliance, Microsoft.Testing.Platform test runner support in `dotnet-remote-testing`, clarified yolo/auto approval behavior in `git-remote-pr`, and visual skill identification with hero images. + +### Changed + +- `git-remote-release` em-dash prohibition now scoped to authored prose only, preserving exact source titles and original formatting while rejecting em-dash usage in generated bullets and release highlight text, +- `git-remote-release` GitHub alert blocks restricted to supported markers (`[!NOTE]`, `[!WARNING]`, `[!IMPORTANT]`, `[!TIP]`, `[!CAUTION]`) with verification rejecting unsupported `[!...]` patterns and loose bracket content, +- `git-remote-release` validation and test coverage expanded with tighter punctuation-only bullet rejection, refined alert-block content validation, and comprehensive test cases covering format violations, alert-marker restrictions, and em-dash scoping, +- `collect-release-evidence.py` and `test-release-evidence.py` enhanced with stricter validation for alert-marker support, punctuation-only bullets, and em-dash presence in authored prose while tolerating source titles unchanged, +- `dotnet-remote-testing` now supports Microsoft.Testing.Platform (MTP) test runner discovered from `global.json` `test.runner` configuration, with MTP-aware command-line argument handling, TRX result collection, filtering support when modules provide it, and extension-capability detection for coverage and diagnostics, +- `dotnet-remote-testing` SKILL.md updated with new "Microsoft Testing Platform and xUnit versions" section documenting MTP discovery, extension dependencies, command-line argument differences from VSTest, xUnit v3/4.x and Codebelt.Extensions.Xunit v12.x version semantics, and troubleshooting for missing dependencies and zero-test scenarios, +- `dotnet-remote-testing` `--filter` option documentation clarified for MTP mode, noting that built modules must support `--filter` and xUnit v3 package 4.0+ provides this syntax, +- `dotnet-remote-testing` failure codes 11 and 12 refined to reflect MTP test-discovery behavior: code 11 now indicates test host failed or discovered zero tests, and code 12 covers missing or unreadable results, +- `git-remote-pr` yolo/auto approval behavior clarified in SKILL.md, AGENTS.md, and eval cases: same-request `yolo`/`auto` skips the approval wait but still shows the plan as status only, with no approval phrase required or awaited, +- README.md updated with clarified `git-remote-pr` approval behavior for yolo/auto mode and `dotnet-remote-testing` MTP support, hero images added for `git-remote-pr` and `dotnet-nuget-update` skills for visual identification in documentation and skill catalogs. + ## [0.11.0] - 2026-09-24 This is a minor release introducing `git-remote-pr`, a Git and GitHub CLI skill for managing GitHub pull requests from the complete committed branch comparison, establishing working-tree scratch isolation standards for all skills, and applying markdown prose formatting conventions across the skill documentation suite. The new skill brings deterministic evidence collection, safe push and assignment handling, post-write verification, upstream tracking validation with branch-name integrity checking, PR approval integrity validation with SHA256 hash computation, and comprehensive regression tests. Repository guidance clarifies where temporary artifacts belong, and skill documentation now follows consistent markdown formatting standards. The `agent-smith` skill is restructured around an operating model framework with a refined decision precedence from 5 to 4 levels, new capability references for agentic engineering patterns and automation, an enhanced validation test suite with regression isolation support, and visual identification assets for all skills. @@ -730,6 +747,7 @@ This is a minor release that introduces two complementary git workflow skills, e - Improved scaffold fidelity with hidden `.bot` asset preservation, explicit UTF-8 and BOM handling, and checks aimed at preventing mojibake or incomplete generated output. +[0.11.1]: https://github.com/codebeltnet/agentic/compare/v0.11.0...v0.11.1 [0.11.0]: https://github.com/codebeltnet/agentic/compare/v0.10.1...v0.11.0 [0.10.1]: https://github.com/codebeltnet/agentic/compare/v0.10.0...v0.10.1 [0.10.0]: https://github.com/codebeltnet/agentic/compare/v0.9.1...v0.10.0 diff --git a/README.md b/README.md index bb18ee4..3b42189 100644 --- a/README.md +++ b/README.md @@ -154,8 +154,8 @@ When repo-managed skills author Markdown, each prose paragraph and list item sta | [dotnet-new-app-slnx](skills/dotnet-new-app-slnx/SKILL.md) | Scaffold a new .NET standalone application solution following codebeltnet engineering conventions. Supports Console, Web, and Worker host families with Startup or Minimal hosting patterns; Web expands into Empty Web, Web API, MVC, or Web App / Razor, plus functional tests and a simplified CI pipeline. | | [trunk-first-repo](skills/trunk-first-repo/SKILL.md) | Initialize a git repository following [scaled trunk-based development](https://trunkbaseddevelopment.com/#scaled-trunk-based-development). Seeds an empty `main` branch, creates a versioned feature branch (`v0.1.0/init`), confirms configured remotes in its post-init summary, and supports a guarded later `push remote ` mode that checks the feature-branch/empty-main state before pushing `main` ahead of the first feature branch so content still reaches main only through peer-reviewed pull requests. | | [dotnet-strong-name-signing](skills/dotnet-strong-name-signing/SKILL.md) | Generate a strong name key (`.snk`) file for signing .NET assemblies using pure .NET cryptography — no Visual Studio Developer PowerShell or `sn.exe` required. Works in any terminal. Defaults to 1024-bit RSA (matching `sn.exe`), with 2048 and 4096 available as options. | -| [git-remote-release](skills/git-remote-release/SKILL.md) | Generate GitHub release notes by summarizing all commits and pull requests between two Git tags or branches in a remote GitHub repository. Accepts a compare URL or separate owner/repo, previous ref, and current ref values; falls back to comparing the current branch against the upstream default branch when no input is provided. Produces a human-friendly `## What's Changed` summary with optional GitHub alert blocks, a `Sources:` section preferring verified merged PRs over covered commits and crediting all verified PR commit contributors with exact GitHub logins, and a full changelog compare link. | -| [git-remote-pr](skills/git-remote-pr/SKILL.md) | Create or refresh a GitHub PR from the complete committed `base...head` changeset. Read-only preparation links a Markdown preview containing the exact proposed title and complete description, assignee, push need, and planned writes before approval; normal execution requires the exact displayed `approve APR-...` phrase and rechecks the write intent, approval-bound normalized review hash, and final preview hash. Material drift ends the approval transaction without regeneration; an explicit refresh starts a new workspace and approval ID. Same-request `yolo`/`auto` proceeds with only the just-presented ID. Existing PR bodies are rebuilt from the current final text diff into concise thematic summaries, with binary assets represented only by paths, statuses, and metadata. Case-insensitive theme coverage reports all mismatches together, and failures remain visible in noninteractive hosts. Draft corrections reuse one workspace. Dirty worktrees, missing or mismatched upstream tracking, divergent branches, and failed post-write checks block completion. Uses Git and `gh`, without Copilot or a browser. | +| [git-remote-release](skills/git-remote-release/SKILL.md) | Generate GitHub release notes by summarizing all commits and pull requests between two Git tags or branches in a remote GitHub repository. Accepts a compare URL or separate owner/repo, previous ref, and current ref values; falls back to comparing the current branch against the upstream default branch when no input is provided. Produces a human-friendly `## What's Changed` section whose first summary line begins `This release `, follows it with curated dash bullets using bold lead-ins and actual explanatory prose, optionally inserts supported GitHub alert blocks, preserves exact verified `Sources:` entries with contributor-complete GitHub logins, keeps em-dash bans scoped to authored prose rather than verbatim source titles, and ends with the full changelog compare link. | +| [git-remote-pr](skills/git-remote-pr/SKILL.md) | Create or refresh a GitHub PR from the complete committed `base...head` changeset. Read-only preparation links a Markdown preview containing the exact proposed title and complete description, assignee, push need, and planned writes before approval; normal execution requires the exact displayed `approve APR-...` phrase and rechecks the write intent, approval-bound normalized review hash, and final preview hash. Material drift ends the approval transaction without regeneration; an explicit refresh starts a new workspace and approval ID. Same-request `yolo`/`auto` still shows the preview but skips the approval wait, executing only the just-presented ID. Existing PR bodies are rebuilt from the current final text diff into concise thematic summaries, with binary assets represented only by paths, statuses, and metadata. Case-insensitive theme coverage reports all mismatches together, and failures remain visible in noninteractive hosts. Draft corrections reuse one workspace. Dirty worktrees, missing or mismatched upstream tracking, divergent branches, and failed post-write checks block completion. Uses Git and `gh`, without Copilot or a browser. | | [dotnet-change-impact](skills/dotnet-change-impact/SKILL.md) | Classify .NET library or NuGet package changes and recommend the correct release bump — `Major`, `Minor`, or `Patch` — for both Semantic Versioning (`MAJOR.MINOR.PATCH`) and .NET assembly/file versioning (`Major.Minor.Build.Revision`), grounded in Microsoft's official .NET compatibility rules. Uses the current Git branch by default when no explicit change details or compare range are provided, resolving it against the upstream/default base branch with local read-only git state. Always returns structured behavioral/binary/source/design-time/backwards compatibility reasoning with the recommendation, even when the bump is clear. | | [dotnet-nuget-update](skills/dotnet-nuget-update/SKILL.md) | Audits and updates NuGet dependencies in .NET repositories with complete declaration accounting before any edit. It supports both `Directory.Packages.props` and project-level `PackageReference` versions, preserves XML structure and line endings, deduplicates package IDs, resolves independent live or offline flat-container version feeds with bounded parallel lookups and per-process memoization, uses bounded network timeouts, and merges results deterministically. Its single-process update runner keeps the audit, in-memory safe-update plan, and structural apply together for fast yolo passes. It keeps stable pins on stable candidates unless prerelease intent is explicit, and applies the TFM-band rule so conditional `net9`/`net10` package declarations stay within their matching major when that major is the compatibility signal rather than jumping to the newest overall release. Normal mode auto-applies revision/patch/minor and same-major prerelease updates, then batches majors for one approval decision; yolo mode applies only the auto classes and reports held majors without asking. | | [dotnet-docfx-digest](skills/dotnet-docfx-digest/SKILL.md) | Create and maintain developer-friendly DocFX documentation for .NET public APIs, including repo-wide no-input audits that inspect source, tests, DocFX config, DocFX `build.content` and `build.overwrite` Markdown inputs, namespace pages, and availability includes before asking for clarification, while treating bare direct skill invocations as autonomous repo-wide runs rather than human-driven checkpoint sessions. Enforces the workflow with two bundled .NET 10 file-based scripts resolved from the loaded skill directory, falling back to the repo-managed source path only when present: `scripts/agents.cs` writes an idempotent, marker-bounded DocFX maintenance block into the repository `AGENTS.md`; `scripts/docfx.cs` is **fast and build-free by default** — it validates Markdown, prose, DocFX overwrite layout, namespace overview pages, `Extension Members` tables, decorated receiver signatures such as `IDecorator`, generic method displays such as `As`, purpose-first summaries, and required per-type/extension examples without invoking `dotnet`, `msbuild`, `docfx`, or `gh`, discovering the public API from existing DocFX YAML metadata or a conservative source scan and ending every run with a `[processes] dotnet=0 msbuild=0 docfx=0 gh=0` summary plus per-phase timings. Compilation and network access are strictly opt-in: `--validate-samples` compiles each C# sample in an isolated project while batching all sample projects into one temporary `.slnx` graph build with bounded MSBuild parallelism and scoped references, `--build-api-model` (alias `--strict-api-discovery`) does reflection-backed discovery from compiled metadata via `MetadataLoadContext` through a single scoped `.slnx` graph build, `--verify-docfx-build` runs the DocFX CLI in a temp copy, and `--search-examples` runs `gh` code search. Final verification adapts to available processors and memory, overlaps isolated DocFX work on high-capacity machines, uses a 30-minute child timeout, and emits 10-second `stderr` heartbeats with active phase, workload, runner count, PID, elapsed time, last-output age, and current child output while preserving machine-readable JSON on `stdout`. Honors a single DocFX metadata `TargetFramework` when `--framework` is omitted, collapses C# 14 extension-block compiler containers such as `$...` back to the authored outer static class in both fast DocFX-YAML discovery and build-backed reflection discovery, validates namespace fly-ins that explain the problem solved/when to use/where to start plus example fly-ins before every C# fence, the Codebelt namespace-and-type-folder overwrite layout (`.docfx/api/namespaces/**/*.md` and `.docfx/api/types/**/*.md` under `build.overwrite` only), keeps `--changed-only` validation scoped to affected docs and APIs while still including brand-new untracked overwrite Markdown, uses the root Codebelt `.snk` when present and falls back to `-p:SkipSignAssembly=true` for keyless strong-name build verification, drains child stdout and stderr concurrently to avoid verbose-build deadlocks, writes deterministic `--assessment-queue` Markdown work queues for noisy audits, preserves working URL references unless a verified HTTP 404 justifies removal, treats unexpected new repo-root or DocFX-workspace files that are not known `dotnet-docfx-digest` deliverables as blocking cleanup diagnostics, keeps assessment/manifests/captured output/helper scripts in temp or session storage instead of the target repository, requires a namespace-first pass across the active queue before net-new type/example authoring during full audits, keeps deeper `EXTENSION_METHOD_MISSING` and `EXTENSION_METHOD_SIGNATURE_MISSING` follow-on diagnostics in that same namespace-layer table-repair phase when they appear after `EXTENSION_SECTION_MISSING` drops, preserves existing BOM and line-ending state while flagging actual mojibake instead of creating encoding-only diffs, and leaves generated DocFX YAML metadata untouched unless `--clean-generated-metadata` is explicitly requested (which runs only after the API model is built, never deleting metadata the run relied on). Documents public API only, uses bundled reference docs for overwrite rules, workflow details, and script behavior, keeps authored API overwrite Markdown under `.docfx/api/namespaces/` and `.docfx/api/types/`, moves legacy authored `.docfx/api/*.md` overwrite files there instead of widening the glob to `api/**/*.md`, teaches namespace and API prose to orient newcomers around purpose instead of inventorying contents, prefers inline or small sibling-batch prose repairs over slow per-page worker fan-out, makes examples start from package-ID usage evidence before type/member-only searches and requires each example to introduce the consumer task before the code, allows multi-type Microsoft Learn-style scenario samples when they better explain the consumer workflow, keeps extension-method examples on readable declaring-class type pages under `.docfx/api/types/` instead of synthetic method-UID filenames or namespace pages that mix extra `uid:` / `example:` blocks into the overview, flags weak skip-compile reasons, requires deterministic `.docfx/skip-compile-allowlist.json` entries for any pre-existing approved skip waivers, treats newly introduced or unallowlisted skip markers as fail-level diagnostics that do not suppress compilation, establishes reflection-backed packets with `--build-api-model --project-manifest` before full-run authoring, forces mid-audit continuations to name that manifest or the sequential assessment/namespace-first fallback explicitly, requires those continuations to restate the fast `docfx.cs --json` rerun cadence, the exact final `docfx.cs --build-api-model --validate-samples --verify-docfx-build --json` gate, and the clean JSON completion contract instead of generic “verify later” prose, treats batch size only as rerun cadence rather than permission to stop, runs a completion repair loop that treats every diagnostic as active work regardless of age or volume, treats newly surfaced follow-on diagnostics as the next repair queue instead of a stop point, reruns packet discovery with `--build-api-model --project-manifest` when fast source-scan packets are unnamed or zero-project, falls back to sequential namespace-first or assessment work queue order when packet discovery is still unusable, treats `EXAMPLE_MISSING`, `EXAMPLE_LEAD_MISSING`, `EXAMPLE_ADVANCED_LEAD_MISSING`, `FAMILY_ANCHOR_EXAMPLE_MISSING`, `SAMPLE_STRUCTURE_INVALID`, `FAIL_NEW_SKIP_MARKER_INTRODUCED`, `SAMPLE_SKIP_NOT_ALLOWLISTED`, and `INTERIM_ARTIFACT_IN_WORKTREE` queues as core work rather than checkpoints or quality backlog, drives large example and lead queues through a concrete fast-path micro-loop (next item or next 3-5 items → rerun → continue), suppresses progress-table/checkpoint output until the completion contract is clean or a real external blocker is reported, treats premature completion-shaped handoffs as execution-protocol failures while the queue is still dirty, reserves the final `--build-api-model --validate-samples --verify-docfx-build` verification for the real end of the queue, exposes `summary.fullVerificationRan`, `summary.canClaimCompletion`, `summary.remainingWorkItems`, `summary.remainingDiagnosticsByCode`, `summary.newlyIntroducedSkipMarkers`, and `summary.interimArtifacts` as machine-readable final gates, reruns the fast `docfx.cs --json` after edits until the queue is empty, then runs the build-backed verification before completion, preserves manual edits and authored Markdown during cleanup, skips recursive generated-output cleanup when a target directory contains documentation or source files, and returns deterministic exit codes plus `--json` reports (including process counts, phase timings, warning counts, and skip-marker accounting) so CI can gate on real failures instead of AI claims. | @@ -621,9 +621,9 @@ Most repositories start with `git init` followed by committing everything direct ### Why git-remote-pr? -**git-remote-pr** prepares a reviewable pull request from every committed change between the resolved base branch and the current head. It inventories commits, changed paths, renames, binaries, and the final patch, then collapses that evidence into a concise opening paragraph followed by bold thematic headings with trailing colons and compact bullets. Related helpers, validation cases, references, formatting changes, and assets become shared reviewer-facing outcomes; complete internal file coverage does not require a prose inventory. The planner rejects malformed default bodies before preview: exactly one opening paragraph, bold colon-ended themes, dash bullets under every theme, and no concluding summary. A deterministic approval ID binds the complete prepared write intent plus the SHA-256 hash of the complete preview with only its two generated approval-ID fields normalized to `{{APPROVAL_ID}}`. A separate SHA-256 hash checks the exact final `preview.md`, including its approval instruction; changing that hash cannot authorize altered preview text. Normal mode requires `approve APR-...` for that preview; generic approval cannot execute or regenerate it. State drift permanently invalidates the transaction and preserves its artifacts. Only an explicit refresh request starts fresh preparation in a new workspace with a new approval ID. Repository templates retain precedence. It uses `git` and `gh` without GitHub Copilot or browser assistance. +**git-remote-pr** prepares a reviewable pull request from every committed change between the resolved base branch and the current head. It inventories commits, changed paths, renames, binaries, and the final patch, then collapses that evidence into a concise opening paragraph followed by bold thematic headings with trailing colons and compact bullets. Related helpers, validation cases, references, formatting changes, and assets become shared reviewer-facing outcomes; complete internal file coverage does not require a prose inventory. The planner rejects malformed default bodies before preview: exactly one opening paragraph, bold colon-ended themes, dash bullets under every theme, and no concluding summary. A deterministic approval ID binds the complete prepared write intent plus the SHA-256 hash of the complete preview with only its two generated approval-ID fields normalized to `{{APPROVAL_ID}}`. A separate SHA-256 hash checks the exact final `preview.md`, including its approval instruction; changing that hash cannot authorize altered preview text. Normal mode requires `approve APR-...` for that preview; generic approval cannot execute or regenerate it. Same-request `yolo`/`auto` still shows the preview but treats that approval phrase as status only, continuing without asking for or waiting on another reply. State drift permanently invalidates the transaction and preserves its artifacts. Only an explicit refresh request starts fresh preparation in a new workspace with a new approval ID. Repository templates retain precedence. It uses `git` and `gh` without GitHub Copilot or browser assistance. -Normal requests stop after showing the proposed title, reviewer-oriented description summary, assignee, push requirement, and exact remote writes. A `yolo` or `auto` attached to the same PR request shows that preview and continues. Existing open PRs are refreshed from the current complete changeset, and the coverage checks still require every changed file to map to a theme represented in the body even when many files collapse into a few sections. Execution refuses plans whose approved title, ready/draft state, body, evidence, or repository snapshot changed after approval. Post-write checks verify the title, body, draft state, assignment, head SHA, and GitHub file inventory. Dirty worktrees, missing upstream tracking, renamed-branch tracking mismatches, and divergent remote branches block the workflow instead of silently retargeting the PR head. Commit creation remains a separate `git-visual-commits` request; release-note writing remains `git-remote-release`. +Normal requests stop after showing the proposed title, reviewer-oriented description summary, assignee, push requirement, and exact remote writes. A `yolo` or `auto` attached to the same PR request still shows that preview, but it does not ask for or wait on `approve APR-...`; it proceeds immediately with the just-presented plan. Existing open PRs are refreshed from the current complete changeset, and the coverage checks still require every changed file to map to a theme represented in the body even when many files collapse into a few sections. Execution refuses plans whose approved title, ready/draft state, body, evidence, or repository snapshot changed after approval. Post-write checks verify the title, body, draft state, assignment, head SHA, and GitHub file inventory. Dirty worktrees, missing upstream tracking, renamed-branch tracking mismatches, and divergent remote branches block the workflow instead of silently retargeting the PR head. Commit creation remains a separate `git-visual-commits` request; release-note writing remains `git-remote-release`. ### Why git-remote-release? @@ -631,17 +631,18 @@ Writing release notes is tedious. Raw commit logs are too noisy, PR titles often **git-remote-release** reads all commits and pull requests between two tags or branches in a remote GitHub repository and produces a polished, paste-ready release note. -Its bundled Python collector uses authenticated `gh` GET requests to expand squash merges into original PR commits, verify commit and file counts, and generate source lines with every recorded contributor. Draft verification rejects missing contributors and incomplete evidence. The model writes the summary from the final file changes and original commit messages, then checks that it covers the meaningful changes. Collection and source verification are deterministic and never invoke a model. +Its bundled Python collector uses authenticated `gh` GET requests to expand squash merges into original PR commits, verify commit and file counts, and generate source lines with every recorded contributor. Draft verification rejects missing contributors, incomplete evidence, malformed `This release ...` openings, missing highlight bullets, punctuation-only highlights, plain blockquotes or unsupported alert markers after the summary, and em dashes in authored release-note prose while still preserving exact source titles. The model writes the summary from the final file changes and original commit messages, then checks that it covers the meaningful changes. Collection and source verification are deterministic and never invoke a model. - **Remote-first workflow** that works entirely through GitHub's API, with no local clone required, - **Compare URL awareness** where a pasted GitHub compare URL is used to extract the owner, repository, and both tags, - **Pull request-preferred analysis** that uses rich PR metadata when available and gracefully falls back to raw commits, - **Default-branch-aware comparisons** that resolve the upstream base and collect only commits on the current branch, - **Effect-oriented summaries** that inspect original PR commits and final diffs so later changes survive stale automated PR descriptions, +- **Release-story structure** where the summary always opens with `This release ...` and then shifts into curated bold-lead highlight bullets, - **Thematic grouping** where related changes are discussed together instead of listed chronologically, -- **GitHub alert blocks** that use `NOTE`, `TIP`, `IMPORTANT`, `WARNING`, and `CAUTION` alerts sparingly and only when the release data supports the attention level, +- **GitHub alert blocks** that use only supported `NOTE`, `TIP`, `IMPORTANT`, `WARNING`, and `CAUTION` markers sparingly and only when the release data supports the attention level, - **Source preservation** where verified merged PRs replace their covered commits, all verified PR commit contributors share the source line with exact GitHub logins, and unresolved lookups are reported as incomplete, -- **Strict format** that always starts with `## What's Changed` and always ends with the full changelog compare link, +- **Strict format** that always starts with `## What's Changed`, forbids em dashes in the authored release-note prose while preserving verbatim source titles, and always ends with the full changelog compare link, - **No invented claims** so every statement in the summary is backed by the commits and pull requests collected, - **Read-only operation** that never mutates repository state. diff --git a/skills/dotnet-nuget-update/SKILL.md b/skills/dotnet-nuget-update/SKILL.md index 877c8bf..777243c 100644 --- a/skills/dotnet-nuget-update/SKILL.md +++ b/skills/dotnet-nuget-update/SKILL.md @@ -12,6 +12,8 @@ description: > # .NET NuGet Update +![.NET NuGet Update](https://raw.githubusercontent.com/codebeltnet/agentic/main/skills/dotnet-nuget-update/assets/hero.jpg) + Use this skill when a .NET repository needs a complete dependency audit or a controlled package update pass. ## Start with the audit, not intuition diff --git a/skills/dotnet-nuget-update/assets/hero.jpg b/skills/dotnet-nuget-update/assets/hero.jpg new file mode 100644 index 0000000..39ec53f Binary files /dev/null and b/skills/dotnet-nuget-update/assets/hero.jpg differ diff --git a/skills/dotnet-remote-testing/SKILL.md b/skills/dotnet-remote-testing/SKILL.md index 940191d..7076fcd 100644 --- a/skills/dotnet-remote-testing/SKILL.md +++ b/skills/dotnet-remote-testing/SKILL.md @@ -139,7 +139,7 @@ dotnet run --file "/scripts/remote-test.cs" -- run --repo-root "` — a specific solution or project (relative to the source root). When omitted, the runner resolves a root solution, a single solution, or a single project automatically. -- `--filter ` — a `dotnet test --filter` expression (test class, trait, etc.). +- `--filter ` — a `dotnet test --filter` expression (test class, trait, etc.). In MTP mode the built modules must support `--filter`; xUnit v3 package 4.0+ supports this syntax. - `--test ` — shortcut for a fully-qualified-name filter (a single test or class). - `-c, --configuration ` — build configuration. - `-f, --framework ` — restrict a multi-targeted test project to one TFM. This also narrows environment resolution, so the run lands on that TFM's single-SDK channel instead of a multi-SDK runner. @@ -148,6 +148,14 @@ Common scoping options (pass through only what the developer asked for): The runner establishes an isolated staged workspace (so container builds never leave Linux `bin`/`obj` in the working tree), mounts a persistent NuGet cache outside the repository, pins the image to its digest, runs restore → build → test, collects TRX results, and removes all transient Docker resources afterward. You do not manage any of that. +### Microsoft Testing Platform and xUnit versions + +The source root's `global.json` selects the command dialect. With `test.runner` set to `Microsoft.Testing.Platform`, the runner uses named `--project`/`--solution` arguments and probes the built modules' help for TRX, filtering, and coverage support. It uses MTP extension arguments directly, without a VSTest `--logger`, `--collect`, or an extra `--` separator. Without that opt-in, it retains the VSTest command path. It never rewrites `global.json` or installs extensions to force compatibility. + +xUnit.net **v3** is the framework family; **4.x** is its package version. Codebelt.Extensions.Xunit **12.x** follows that package generation and MTP v2. The runner selects behavior from configuration and available capabilities, not those version numbers. Missing TRX reporting or requested coverage/filter support fails explicitly. A zero exit with missing TRX results or zero discovered tests is never reported as a passing suite. + +For a CI failure with `Zero tests ran`, inspect the exact command and extension options before changing tests or dependencies. MTP exit code 5 indicates invalid command-line arguments; `--show-log` includes stdout and stderr, and the normal failure summary preserves the zero-test message and exit code. For example, CI's `--hangdump` options require the separate `Microsoft.Testing.Extensions.HangDump` package; neither xUnit nor a coverage package implies that extension is installed. Report missing dependencies when only remote execution was requested; change them only under a separate repair request. + Two diagnostic options exist for when a result needs explaining, not for routine runs: - `--show-log` — print the container's full restore/build/test log. Use it when the summarized failure detail is not enough, never by default. @@ -231,8 +239,8 @@ A run is reproducible in terms of environment, requested image, resolved digest, | `8` | `SourceStaging` | Workspace could not be staged | Report it as infrastructure | | `9` | `Restore` | `dotnet restore` failed in the container | Report the restore error, not "tests failed" | | `10` | `Compilation` | Build failed in the container | Report the compiler errors, not "tests failed" | -| `11` | `TestHost` | Test host crashed | Report as infrastructure with the output tail | -| `12` | `ResultProcessing` | Results unreadable | Report it; results are unknown, not passing | +| `11` | `TestHost` | Test host failed or discovered zero tests | Report as infrastructure with the output tail | +| `12` | `ResultProcessing` | Results missing or unreadable | Report it; results are unknown, not passing | | `13` | `Cleanup` | Transient resources left behind | Report the exact identifiers the runner names | | `14` | `Cancelled` | Timeout or interrupt | Report how far it got | | `15` | `ReleaseMetadataUnavailable` | Release index unreachable, no cache | Report it; suggest `--offline --cache-root` or a named environment | diff --git a/skills/dotnet-remote-testing/evals/evals.json b/skills/dotnet-remote-testing/evals/evals.json index 92412e1..fc30880 100644 --- a/skills/dotnet-remote-testing/evals/evals.json +++ b/skills/dotnet-remote-testing/evals/evals.json @@ -241,6 +241,41 @@ "evals/files/configured/test/Api.Tests/Api.Tests.csproj", "evals/files/configured/test/Api.Tests/HealthTests.cs" ] + }, + { + "id": 15, + "prompt": "Run the ShouldRunOnLinux test in Ubuntu with coverage. This repo uses Codebelt xUnit 12 and xunit.v3 package 4.", + "expected_output": "The runner honors global.json MTP mode, executes the selected test in Docker, and reports exactly one passing test from TRX. Filtering and installed Coverlet support are discovered from built-module help.", + "expectations": [ + "Runs through the bundled runner with --test ShouldRunOnLinux and --coverage", + "Uses MTP named project selection and supported reporting, filtering, and coverage options", + "Reports exactly one passing Linux test and no skipped or failing tests", + "Does not modify global.json, packages, or the configured Ubuntu image", + "Never substitutes host testing or treats zero tests as success" + ], + "files": [ + "evals/files/mtp-xunit/global.json", + "evals/files/mtp-xunit/testenvironments.json", + "evals/files/mtp-xunit/test/Mtp.Tests/Mtp.Tests.csproj", + "evals/files/mtp-xunit/test/Mtp.Tests/PlatformTests.cs" + ] + }, + { + "id": 16, + "prompt": "Remote test this xUnit v3 package 4 repository in Ubuntu and explain any failure.", + "expected_output": "The runner executes both tests using MTP and returns test-failure exit code 1, with one passing test and the intentional assertion failure preserved in its TRX summary.", + "expectations": [ + "Reports one passed and one failed test, with the intentional assertion message", + "Names Mtp.Tests.PlatformTests.ShouldReportAnAssertionFailure and net10.0", + "Distinguishes the assertion failure from a test-host or reporting failure", + "Does not edit the failing test or rerun on the host" + ], + "files": [ + "evals/files/mtp-xunit/global.json", + "evals/files/mtp-xunit/testenvironments.json", + "evals/files/mtp-xunit/test/Mtp.Tests/Mtp.Tests.csproj", + "evals/files/mtp-xunit/test/Mtp.Tests/PlatformTests.cs" + ] } ] } diff --git a/skills/dotnet-remote-testing/evals/files/mtp-xunit/global.json b/skills/dotnet-remote-testing/evals/files/mtp-xunit/global.json new file mode 100644 index 0000000..3140116 --- /dev/null +++ b/skills/dotnet-remote-testing/evals/files/mtp-xunit/global.json @@ -0,0 +1,5 @@ +{ + "test": { + "runner": "Microsoft.Testing.Platform" + } +} diff --git a/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/Mtp.Tests.csproj b/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/Mtp.Tests.csproj new file mode 100644 index 0000000..e00ebfc --- /dev/null +++ b/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/Mtp.Tests.csproj @@ -0,0 +1,15 @@ + + + net10.0 + enable + Exe + true + false + true + + + + + + + diff --git a/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/PlatformTests.cs b/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/PlatformTests.cs new file mode 100644 index 0000000..1db7c81 --- /dev/null +++ b/skills/dotnet-remote-testing/evals/files/mtp-xunit/test/Mtp.Tests/PlatformTests.cs @@ -0,0 +1,13 @@ +using System; +using Xunit; + +namespace Mtp.Tests; + +public class PlatformTests +{ + [Fact] + public void ShouldRunOnLinux() => Assert.True(OperatingSystem.IsLinux()); + + [Fact] + public void ShouldReportAnAssertionFailure() => Assert.Fail("Intentional failure to verify remote TRX reporting."); +} diff --git a/skills/dotnet-remote-testing/evals/files/mtp-xunit/testenvironments.json b/skills/dotnet-remote-testing/evals/files/mtp-xunit/testenvironments.json new file mode 100644 index 0000000..6322da2 --- /dev/null +++ b/skills/dotnet-remote-testing/evals/files/mtp-xunit/testenvironments.json @@ -0,0 +1,10 @@ +{ + "version": "1", + "environments": [ + { + "name": "Docker-Ubuntu", + "type": "docker", + "dockerImage": "codebeltnet/ubuntu-testrunner:10" + } + ] +} diff --git a/skills/dotnet-remote-testing/references/docker-execution.md b/skills/dotnet-remote-testing/references/docker-execution.md index 4b80773..d1f2643 100644 --- a/skills/dotnet-remote-testing/references/docker-execution.md +++ b/skills/dotnet-remote-testing/references/docker-execution.md @@ -77,7 +77,7 @@ The dependency cache (`/nuget`, via `NUGET_PACKAGES`) is deliberately separated ## In-container phases -The entrypoint runs three ordered phases and emits a machine-readable marker with each phase's exit code: +The entrypoint runs three ordered phases and emits a machine-readable marker with each phase's exit code. For VSTest: ``` dotnet restore @@ -87,6 +87,10 @@ dotnet test -c --no-build [--filter …] [--framework …] `restore` and `build` stop the run on failure; `test` always runs to completion so a TRX is produced even when tests fail. The runner works with the repository's configured .NET testing infrastructure; it does not install or alter test packages, and it does not assume a single testing framework. +When the source root's `global.json` selects `Microsoft.Testing.Platform`, the test phase uses `dotnet test --project ` or `--solution `. After building, it probes `dotnet test ... --no-build --help` inside the container. It selects `--report-trx` when available, otherwise xUnit's built-in `--report-xunit-trx`; absence of both is a failure. Requested coverage selects the installed `--coverlet` or `--coverage` extension with Cobertura output, and requested filtering requires `--filter` support. These are capability choices, never retries after execution failure. MTP arguments are passed directly without a `--` separator. Mixed modules must support the selected options; the runner never skips incompatible projects or installs packages. A framework restriction uses `-p:TargetFramework=` for restore and `--framework ` for build/test. + +Compatibility references: [xUnit v3 package 4.0 release notes](https://xunit.net/releases/v3/4.0.0), [MTP dotnet test command](https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test-mtp), and [MTP troubleshooting](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-troubleshooting). xUnit v3 package 4.0 introduced MTP v2 as the default and VSTest-style `--filter` support; its `--report-xunit-trx` option is unchanged. + ## Test scope Supported through options that pass straight into the phases: entire solution, a project, an individual test or class (`--test`/`--filter`), configuration (`--configuration`), a single TFM of a multi-targeted project (`--framework`), and coverage (`--coverage`, only when the project already supports it). @@ -103,6 +107,8 @@ Pull/restore/build noise is suppressed on success. `--show-log` prints the conta Failures are classified into distinct kinds so a container/infrastructure problem is never misreported as a failing unit test: `Configuration`, `UnsupportedEnvironment`, `DockerUnavailable`, `ImageResolution`, `SdkIncompatibility`, `SourceStaging`, `Restore`, `Compilation`, `TestHost`, `TestFailure`, `ResultProcessing`, `Cleanup`, `Cancelled`, `ReleaseMetadataUnavailable`. A non-zero `dotnet test` exit with a TRX containing failures is a `TestFailure`; a non-zero exit with no failing results (crash, no discovered tests, missing adapter) is a `TestHost` failure. +A zero exit without a completed test phase or any TRX is `ResultProcessing`, and a TRX with zero tests is `TestHost`. Neither establishes a successful run. Test-host summaries preserve the output tail (including `Zero tests ran` and the process exit code); `--show-log` includes both captured output streams. + Each phase emits a machine-readable end marker, so the log between two markers is exactly that phase's output. An infrastructure failure is reported with *its own* phase's log rather than a tail of everything — a build failure names the offending file and compiler error instead of trailing test-runner chatter. Any failing tests already recorded in a TRX are reported first even when the phase failed for another reason, so an assertion failure followed by a test-host crash does not disappear behind the crash. ## Cancellation and cleanup diff --git a/skills/dotnet-remote-testing/scripts/remote-test.cs b/skills/dotnet-remote-testing/scripts/remote-test.cs index 7df7f5a..004d148 100644 --- a/skills/dotnet-remote-testing/scripts/remote-test.cs +++ b/skills/dotnet-remote-testing/scripts/remote-test.cs @@ -1335,6 +1335,7 @@ internal sealed record ContainerMount(string HostPath, string ContainerPath, boo internal sealed record TestCommandOptions { + public bool MicrosoftTestingPlatform { get; init; } public string? Target { get; init; } public string Configuration { get; init; } = "Debug"; public string? Framework { get; init; } @@ -1359,6 +1360,25 @@ internal static class ContainerPlanner { internal const string PhaseMarkerPrefix = "##RT_PHASE_END:"; + // Match the SDK runner selection in the staged workspace; never infer it from package names. + public static bool UsesMicrosoftTestingPlatform(string sourceRoot) + { + var path = Path.Combine(sourceRoot, "global.json"); + if (!File.Exists(path)) + { + return false; + } + + using var json = JsonDocument.Parse(File.ReadAllText(path), new JsonDocumentOptions + { + AllowTrailingCommas = true, + CommentHandling = JsonCommentHandling.Skip, + }); + return json.RootElement.TryGetProperty("test", out var test) + && test.TryGetProperty("runner", out var runner) + && runner.GetString() == "Microsoft.Testing.Platform"; + } + // Resolve the dotnet test --filter expression. --test is sugar for a FullyQualifiedName contains match. public static string? ResolveFilter(string? filter, string? test) { @@ -1398,9 +1418,39 @@ public static string BuildEntrypoint(TestCommandOptions o) sb.Append("umask 000\n"); sb.Append($"cd {Shell.Quote(o.WorkDir)} || {{ echo '{PhaseMarkerPrefix}staging:1##'; exit 8; }}\n"); sb.Append("run_phase() { name=\"$1\"; shift; \"$@\"; code=$?; echo \"" + PhaseMarkerPrefix + "${name}:${code}##\"; return $code; }\n"); - sb.Append($"run_phase restore dotnet restore{target}{fw} || exit 9\n"); + var restoreFramework = string.IsNullOrWhiteSpace(o.Framework) ? "" : $" -p:TargetFramework={Shell.Quote(o.Framework)}"; + sb.Append($"run_phase restore dotnet restore{target}{restoreFramework} || exit 9\n"); sb.Append($"run_phase build dotnet build{target}{cfg}{fw} --no-restore || exit 10\n"); - sb.Append($"run_phase test dotnet test{target}{cfg}{fw} --no-build{filter}{coverage} --results-directory {Shell.Quote(o.ResultsDir)} --logger trx\n"); + if (o.MicrosoftTestingPlatform) + { + var testTarget = string.IsNullOrWhiteSpace(o.Target) ? "" + : $" {(o.Target.EndsWith("proj", StringComparison.OrdinalIgnoreCase) ? "--project" : "--solution")} {Shell.Quote(o.Target)}"; + var command = $"dotnet test{testTarget}{cfg}{fw} --no-build"; + // Extension switches belong to the test applications. Ask the built modules which ones + // they support instead of installing packages or guessing from an xUnit version number. + sb.Append("run_tests() {\n"); + sb.Append($" help=$({command} --help 2>&1) || {{ printf '%s\\n' \"$help\"; return 1; }}\n"); + sb.Append(" supports() { grep -Eq -- \"(^|[[:space:]])$1([[:space:]<]|$)\" <<< \"$help\"; }\n"); + sb.Append(" args=()\n"); + sb.Append(" if supports --report-trx; then args+=(--report-trx); elif supports --report-xunit-trx; then args+=(--report-xunit-trx); else printf '%s\\n' \"$help\" 'No supported MTP TRX reporter. Reference an MTP TRX extension or use the xUnit reporter.'; return 1; fi\n"); + if (!string.IsNullOrWhiteSpace(o.Filter)) + { + sb.Append(" supports --filter || { echo 'The selected MTP modules do not support --filter (xUnit v3 requires package 4.0+).'; return 1; }\n"); + sb.Append($" args+=(--filter {Shell.Quote(o.Filter)})\n"); + } + + if (o.Coverage) + { + sb.Append(" if supports --coverlet; then args+=(--coverlet --coverlet-output-format cobertura); elif supports --coverage; then args+=(--coverage --coverage-output-format cobertura); else echo 'No supported MTP coverage extension is installed.'; return 1; fi\n"); + } + + sb.Append($" {command} --results-directory {Shell.Quote(o.ResultsDir)} \"${{args[@]}}\"\n"); + sb.Append("}\nrun_phase test run_tests\n"); + } + else + { + sb.Append($"run_phase test dotnet test{target}{cfg}{fw} --no-build{filter}{coverage} --results-directory {Shell.Quote(o.ResultsDir)} --logger trx\n"); + } sb.Append("exit $?\n"); return sb.ToString(); } @@ -1928,6 +1978,17 @@ public static ExecutionOutcome Classify( return new ExecutionOutcome(FailureKind.TestFailure, "test", containerExitCode, $"{results.Failed} test(s) failed."); } + if (!testRan || results.TrxFilesParsed == 0) + { + return new ExecutionOutcome(FailureKind.ResultProcessing, "test", containerExitCode, + "The test phase did not produce TRX results. A zero process exit is insufficient to establish a passing run."); + } + + if (results.Total == 0) + { + return new ExecutionOutcome(FailureKind.TestHost, "test", containerExitCode, "Zero tests were discovered. Check the selected target and filter."); + } + return new ExecutionOutcome(FailureKind.None, "test", 0, "All tests passed."); } @@ -3128,6 +3189,7 @@ public static async Task PlanAsync(Options options) private static TestCommandOptions BuildTestOptions(Options options, string sourceRoot) => new() { + MicrosoftTestingPlatform = ContainerPlanner.UsesMicrosoftTestingPlatform(sourceRoot), Target = TargetResolver.Resolve(sourceRoot, options.Project), Configuration = options.Configuration, Framework = options.Framework, @@ -3571,7 +3633,8 @@ private static int EmitRunResult( // would leave the developer with a verdict and no cause. Prefer the failing phase's own log. var phaseLog = FailureClassifier.PhaseOutput(proc.StdOut, outcome.Phase); var detail = FirstNonEmpty( - ErrorLines(phaseLog, 20), LastLines(phaseLog, 40), LastLines(proc.StdErr, 20), LastLines(proc.StdOut, 40)); + outcome.Kind == FailureKind.TestHost ? LastLines(phaseLog, 40) : ErrorLines(phaseLog, 20), + LastLines(phaseLog, 40), LastLines(proc.StdErr, 20), LastLines(proc.StdOut, 40)); if (!string.IsNullOrWhiteSpace(detail)) { Console.Error.WriteLine(); @@ -3587,6 +3650,12 @@ private static int EmitRunResult( Console.WriteLine(proc.StdOut.TrimEnd()); } + if (options.ShowLog && !string.IsNullOrWhiteSpace(proc.StdErr)) + { + Console.Error.WriteLine("--- container stderr ---"); + Console.Error.WriteLine(proc.StdErr.TrimEnd()); + } + if (cleanup.Leftovers.Count > 0) { Console.Error.WriteLine("Cleanup left resources: " + string.Join(", ", cleanup.Leftovers)); @@ -4123,6 +4192,34 @@ private static void CommandPlanningTests() Check("entrypoint honors configuration/framework/filter/coverage", entry.Contains("-c 'Release'") && entry.Contains("--framework 'net10.0'") && entry.Contains("--filter 'Category=Unit'") && entry.Contains("XPlat Code Coverage")); Check("entrypoint writes trx to results mount", entry.Contains("--results-directory '/results'") && entry.Contains("--logger trx")); + Check("restore uses an MSBuild framework property, not the force switch", + entry.Contains("dotnet restore 'test/Foo/Foo.csproj' -p:TargetFramework='net10.0'") + && !entry.Contains("dotnet restore 'test/Foo/Foo.csproj' --framework")); + + var mtpOptions = new TestCommandOptions + { + MicrosoftTestingPlatform = true, + Target = "test/Foo's tests/Foo.csproj", + Configuration = "Release", + Framework = "net10.0", + Filter = "FullyQualifiedName~PassingTest", + Coverage = true, + }; + var mtp = ContainerPlanner.BuildEntrypoint(mtpOptions); + Check("MTP uses explicit project selection with shell quoting", + mtp.Contains("dotnet test --project 'test/Foo'\\''s tests/Foo.csproj' -c 'Release' --framework 'net10.0' --no-build")); + Check("MTP probes built module capabilities before running", + mtp.Contains("--no-build --help 2>&1") && mtp.Contains("supports --report-trx") && mtp.Contains("supports --report-xunit-trx")); + Check("MTP passes supported filter and coverage switches", + mtp.Contains("args+=(--filter 'FullyQualifiedName~PassingTest')") && mtp.Contains("--coverlet-output-format cobertura") && mtp.Contains("--coverage-output-format cobertura")); + Check("MTP never passes VSTest logging or collection arguments", + !mtp.Contains("--logger") && !mtp.Contains("--collect") && !mtp.Contains(" -- --report")); + Check("MTP solution selection uses --solution", + ContainerPlanner.BuildEntrypoint(mtpOptions with { Target = "App.slnx" }).Contains("dotnet test --solution 'App.slnx'")); + Check("MTP retains implicit target discovery", + ContainerPlanner.BuildEntrypoint(mtpOptions with { Target = null }).Contains("dotnet test -c 'Release'")); + Check("MTP does not enable unrequested coverage or filters", + !ContainerPlanner.BuildEntrypoint(mtpOptions with { Filter = null, Coverage = false }).Contains("args+=(--coverlet")); var plan = ContainerPlanner.Build( "mcr.microsoft.com/dotnet/sdk:10.0.302", @@ -4155,6 +4252,14 @@ private static void CommandPlanningTests() File.WriteAllText(Path.Combine(tempRoot, "App.slnx"), ""); Check("root solution preferred over project", TargetResolver.Resolve(tempRoot, null) == "App.slnx"); Check("explicit project overrides discovery", TargetResolver.Resolve(tempRoot, "test/Foo/Foo.csproj") == "test/Foo/Foo.csproj"); + Check("no global.json retains VSTest commands", !ContainerPlanner.UsesMicrosoftTestingPlatform(tempRoot)); + var globalJson = Path.Combine(tempRoot, "global.json"); + File.WriteAllText(globalJson, "{ /* SDK configuration */ \"test\": { \"runner\": \"Microsoft.Testing.Platform\", }, }"); + Check("global.json selects MTP and permits SDK JSON comments", ContainerPlanner.UsesMicrosoftTestingPlatform(tempRoot)); + File.WriteAllText(globalJson, "{\"test\":{\"runner\":\"VSTest\"}}"); + Check("explicit VSTest retains legacy commands", !ContainerPlanner.UsesMicrosoftTestingPlatform(tempRoot)); + File.WriteAllText(globalJson, "{\"sdk\":{\"version\":\"10.0.100\"}}"); + Check("SDK pin alone does not select MTP", !ContainerPlanner.UsesMicrosoftTestingPlatform(tempRoot)); } finally { @@ -4373,6 +4478,10 @@ private static void FailureClassificationTests() FailureClassifier.Classify(new Dictionary { ["restore"] = 0, ["build"] = 0, ["test"] = 1 }, 1, false, TestRunResult.Empty).Kind == FailureKind.TestHost); Check("all passing classified as None", FailureClassifier.Classify(new Dictionary { ["restore"] = 0, ["build"] = 0, ["test"] = 0 }, 0, false, passing).Kind == FailureKind.None); + Check("zero exit without TRX is not success", + FailureClassifier.Classify(new Dictionary { ["test"] = 0 }, 0, false, TestRunResult.Empty).Kind == FailureKind.ResultProcessing); + Check("zero discovered tests is not success", + FailureClassifier.Classify(new Dictionary { ["test"] = 0 }, 0, false, new TestRunResult { TrxFilesParsed = 1 }).Kind == FailureKind.TestHost); Check("cancellation wins over everything", FailureClassifier.Classify(new Dictionary { ["restore"] = 0 }, -1, true, passing).Kind == FailureKind.Cancelled); diff --git a/skills/dotnet-remote-testing/scripts/validate-skill.ps1 b/skills/dotnet-remote-testing/scripts/validate-skill.ps1 index e51fdda..8f4db3c 100644 --- a/skills/dotnet-remote-testing/scripts/validate-skill.ps1 +++ b/skills/dotnet-remote-testing/scripts/validate-skill.ps1 @@ -68,7 +68,9 @@ if ($doThisNow -lt 0) { if ($commands -ge 0 -and $commands -lt $doThisNow) { throw 'SKILL.md lists the command surface before "## Do this now"; the imperative must come first.' } -if ($doThisNow -gt 100) { +# A decorative hero between the title and the imperative is not an intake or workflow section. +$opening = [regex]::Replace($body.Substring(0, $doThisNow), '(?m)^!\[[^\r\n]*\]\([^\r\n]*\)\r?\n', '') +if ($opening.Length -gt 100) { throw "SKILL.md places '## Do this now' too late (body offset $doThisNow); it must lead the document body." } diff --git a/skills/git-remote-pr/SKILL.md b/skills/git-remote-pr/SKILL.md index 5a8a13e..85cf116 100644 --- a/skills/git-remote-pr/SKILL.md +++ b/skills/git-remote-pr/SKILL.md @@ -6,13 +6,15 @@ description: > # Git Remote PR +![Git Remote PR](https://raw.githubusercontent.com/codebeltnet/agentic/main/skills/git-remote-pr/assets/hero.jpg) + Open or maintain one GitHub pull request for the entire committed current branch. Use Git, `gh`, and GitHub APIs. Never use GitHub Copilot, `gh copilot`, browser automation, GitHub's generated PR description, or another model-backed GitHub service. The agent reading this skill writes concise reviewer-oriented prose from collected evidence; bundled scripts collect and verify facts only. Use PowerShell 7 (`pwsh`) on Windows, Linux, and macOS. ## Routing and authorization - Trigger on an explicit request to create, open, make, or refresh a GitHub PR, or `git remote pr`. `/git-remote-pr` may force selection where supported. Bare `yolo` or `auto` never triggers this skill. - The explicit PR request authorizes read-only preparation. In normal mode, show the preview below and **stop for the exact `approve APR-...` phrase displayed in that preview before any remote write**. Approval for Preview A can execute only Plan A. -- `yolo` or `auto` attached to the same explicit PR request authorizes the narrow write phase after displaying the same preview as status. Continue without a second question. Neither mode authorizes force push, rebase, reset, amend, merge, branch deletion, auto-merge, or unrelated writes. +- `yolo` or `auto` attached to the same explicit PR request still requires the full preview, but it skips the approval requirement itself. Show the same preview as status, do not ask for or wait on `approve APR-...`, and continue immediately with that just-presented plan's ID. Neither mode authorizes force push, rebase, reset, amend, merge, branch deletion, auto-merge, or unrelated writes. - A dirty worktree, including untracked files, blocks both modes. Explain that a PR contains committed changes only. Do not stage, commit, stash, discard, or automatically invoke `git-visual-commits`. - A missing upstream tracking branch, or a local/upstream branch-name mismatch, also blocks both modes. The skill never guesses a PR head after a rename or silent tracking drift; fix tracking first, then rerun the workflow. - Commit requests belong to `git-visual-commits`; squash wording belongs to `git-visual-squash-summary`; release notes belong to `git-remote-release`; changelogs belong to `git-keep-a-changelog`. This skill independently reads the complete PR comparison. @@ -71,14 +73,14 @@ Before any `git push`, `gh pr create`, `gh pr edit`, assignee change, or equival - Exact planned writes: optional normal push to the already-tracked head branch, create or edit the intended PR, and add the authenticated assignee if absent. Include ready/draft state. - A prominent `Approval ID: APR-...` and the final instruction `To approve this exact preview, reply: approve APR-...`, using the same ID as `plan.json.approval_id`. -In normal mode give the preview link and its exact approval phrase, then stop. Bare `approved`, `yes`, `go ahead`, or `looks good` does not authorize execution. Re-present the **existing preview link** and ask for the exact displayed phrase; do not regenerate anything or infer, fill in, or manufacture the user's approval ID. With same-request `yolo`/`auto`, give the same preview link and phrase as status, then pass that just-presented plan's ID without waiting for another message. No approval is inferred from a prior unrelated `yolo`/`auto`. +In normal mode give the preview link and its exact approval phrase, then stop. Bare `approved`, `yes`, `go ahead`, or `looks good` does not authorize execution. Re-present the **existing preview link** and ask for the exact displayed phrase; do not regenerate anything or infer, fill in, or manufacture the user's approval ID. With same-request `yolo`/`auto`, give the same preview link and phrase as status only, do not ask for or wait on that phrase, and pass that just-presented plan's ID immediately. No approval is inferred from a prior unrelated `yolo`/`auto`. ## Phase 2: execute and verify -After approval (or the same-request auto modifier), run: +After explicit approval in normal mode, or immediately after the preview in same-request auto mode, run: ```text -pwsh -NoProfile -NonInteractive -File /scripts/execute-pr.ps1 -PlanFile /plan.json -ApprovalId +pwsh -NoProfile -NonInteractive -File /scripts/execute-pr.ps1 -PlanFile /plan.json -ApprovalId ``` The helper matches the supplied approval ID, checks the exact final preview hash, normalizes only the two generated approval-ID slots back to `{{APPROVAL_ID}}`, verifies `review_hash`, and recomputes the approval ID from the complete write intent plus that review hash. It then checks evidence/body hashes and independently rechecks the material snapshot, and fails before a write when any check differs. Changed local `HEAD`, worktree, base, remote head, authenticated user, PR title/body/state, template, changed-file inventory, or approved artifact invalidates approval. diff --git a/skills/git-remote-pr/assets/hero.jpg b/skills/git-remote-pr/assets/hero.jpg new file mode 100644 index 0000000..dfacc71 Binary files /dev/null and b/skills/git-remote-pr/assets/hero.jpg differ diff --git a/skills/git-remote-pr/evals/evals.json b/skills/git-remote-pr/evals/evals.json index 88a6dba..215607e 100644 --- a/skills/git-remote-pr/evals/evals.json +++ b/skills/git-remote-pr/evals/evals.json @@ -16,8 +16,8 @@ { "id": 3, "prompt": "Create a PR yolo from v12.0.2/chore-work. There is a clean branch and no existing PR.", - "expected_output": "Displays the CREATE mutation preview, then performs the permitted writes without a second confirmation; the title is V12.0.2/chore work.", - "expectations": ["A clickable preview.md link containing the complete exact title and body is provided before remote mutations", "Same-request yolo removes only the approval wait", "No force push or history rewrite", "The PR is ready for review unless draft was explicitly requested", "Authenticated GitHub identity is used for assignment, not git user.name"] + "expected_output": "Displays the CREATE mutation preview, then performs the permitted writes without asking for or waiting on `approve APR-...`; the title is V12.0.2/chore work.", + "expectations": ["A clickable preview.md link containing the complete exact title and body is provided before remote mutations", "Same-request yolo treats the preview approval phrase as status only and does not ask for or wait on it", "No force push or history rewrite", "The PR is ready for review unless draft was explicitly requested", "Authenticated GitHub identity is used for assignment, not git user.name"] }, { "id": 4, diff --git a/skills/git-remote-release/SKILL.md b/skills/git-remote-release/SKILL.md index d3466e1..9c6b449 100644 --- a/skills/git-remote-release/SKILL.md +++ b/skills/git-remote-release/SKILL.md @@ -32,7 +32,7 @@ Save the draft beside the evidence and check it before returning: python /scripts/collect-release-evidence.py verify /release-evidence.json /release-notes.md ``` -Fix source omissions, extra source entries, contributor changes, and format errors until verification succeeds. This checks attribution and format, not whether prose covers every change or correctly describes its impact. Incomplete evidence cannot pass this gate; report that limitation rather than claiming verification passed. If Python/`gh` is unavailable, refs exist only locally, or the user supplies an offline snapshot, follow the manual workflow below with the same count and contributor checks. State the collection limitation; do not quietly skip PR commit expansion. Never call an AI service from either script. +Fix source omissions, extra source entries, contributor changes, structural summary errors, and format errors until verification succeeds. This checks exact sources plus the release-note structure (`This release ...`, release-highlight bullets, the em dash prohibition, and final changelog positioning), not whether the narrative fully captures every change or weighs the release perfectly. Incomplete evidence cannot pass this gate; report that limitation rather than claiming verification passed. If Python/`gh` is unavailable, refs exist only locally, or the user supplies an offline snapshot, follow the manual workflow below with the same count and contributor checks. State the collection limitation; do not quietly skip PR commit expansion. Never call an AI service from either script. ## Input @@ -182,6 +182,10 @@ Read through all collected pull requests and commits. Understand what changed, w Read original PR commit messages and the final comparison's changed-file inventory and relevant patches before drafting. Fetch paginated PR files or individual file/commit diffs when the compare response is capped or omits a needed patch. PR titles, automated PR bodies, and package release notes may describe only the initial change and become stale after later commits. Resolve discrepancies against the final diff; use commit messages for intent, and do not describe reverted intermediate work as shipped. Build a compact evidence inventory connecting each meaningful change to its files and commits, then check that the summary covers it. One source URL can represent many changes by several people. Never reduce an entire service PR to dependency updates solely because its title or opening body says so. +> Evidence completeness and summary completeness are different concerns. Inspect and account for all meaningful evidence, but curate the release summary around the important user-facing and maintainer-facing outcomes rather than reproducing the evidence inventory. + +The release summary still needs to account for the full shipped delta, but it should tell the release story instead of giving every evidence item equal narrative weight. + The summary should explain the effect of the changes, not just the implementation. A good release note tells users what they can expect from this version, not just what code was modified. ### Step 5: Compose the release notes @@ -193,7 +197,11 @@ Follow the exact output format defined below. Every release note must start with ```markdown ## What's Changed - +This release . + +- **** , +- **** , +- **** . @@ -207,49 +215,57 @@ Sources: Keep each prose paragraph and Markdown list item on one physical line regardless of length. Do not hard-wrap release notes to a fixed column width; rely on editor soft wrapping and use physical line breaks only between Markdown structures. Rejoin unnecessary hard wraps in any existing prose you edit. -### The summary section +### The release summary contract -The summary is the heart of the release note. It must be: +The summary is the heart of the release note. It must always have two parts in this order: -- **Human-friendly** — written for someone scanning the release to understand what changed -- **Effect-oriented** — explains what users and maintainers can expect, not just what was modified -- **Evidence-backed** — every claim must be supported by the commits or pull requests collected -- **Grouped logically** — related changes are discussed together, not listed chronologically -- **Honest** — no invented impact, no unsupported claims, no vague filler like "various improvements" +1. one concise release-level opening paragraph; +2. one or more curated release-highlight bullets. -For small releases (a handful of changes), prefer a concise paragraph or short bullet list. +#### Opening paragraph -For larger releases, prefer grouped bullets organized by theme: new features, fixes, infrastructure, breaking changes, etc. +The first non-empty summary line after `## What's Changed` must begin exactly with `This release `. Choose a natural verb or phrase from the evidence, such as `introduces`, `strengthens`, `improves`, `refines`, `fixes`, or `streamlines`, but do not force awkward wording just to match an example. -Avoid simply repeating PR titles or commit messages unless they are already clear and release-note friendly. Rewrite them into prose that explains the effect. +The opening paragraph must: -### Key capabilities formatting (when included) +- summarize the release as a whole, +- emphasize the primary purpose and effect, +- stay human-friendly for someone scanning a GitHub release, +- avoid implementation inventories, and +- avoid changelog-style `Added`, `Changed`, or `Fixed` section framing unless those words are naturally required by the subject matter. -When the release note includes a "Key capabilities" section, each bullet must be written as a natural sentence with a bolded lead-in. +Keep it to exactly one concise paragraph on one physical line. -Do not use a bold label followed by an em dash, colon, or definition-style fragment. +#### Release-highlight bullets -Avoid this style: +The opening paragraph must always be followed by one or more dash bullets before any optional alert blocks or the `Sources:` section. -```markdown -- **Thematic grouping** — Related changes are discussed together instead of listed chronologically -``` +These bullets are curated release highlights. They are not a commit log, a changed-file inventory, an exhaustive evidence transcript, or Keep a Changelog sections. Group related implementation details into meaningful outcomes that a maintainer or release reader should care about. -Use this style instead: +Use this style: ```markdown -- **Thematic grouping** where related changes are discussed together instead of listed chronologically, +- **Team catalog matching** now compares the exact team component with `spec.name` case-insensitively, preventing descriptions from being borrowed from similarly prefixed teams, +- **Identity-provider validation** supports configured group prefixes and role suffixes while unmatched teams complete without an incorrect catalog description, +- **Regression coverage** exercises overlapping team names, organization-name changes, legacy IDP prefixes, and additional role suffixes. ``` -The bold text should highlight the capability name, but the full bullet must read as one natural sentence. +Each release-highlight bullet must: -Preferred pattern: +- begin with `- `, +- use a concise bold lead-in, +- continue directly into natural sentence prose with actual explanatory text, not bare punctuation, +- describe an outcome or effect rather than merely naming implementation work, +- remain concise enough to scan, and +- end with `,` except for the final bullet, which ends with `.`. -```markdown -- **** where/that/so/with , -``` +The bold lead-in is not a heading. Do not use `**** — ...`, `****: ...`, or bold-leading prose paragraphs as a substitute for bullets. + +#### Em dash prohibition -End each bullet with `,` except the final bullet in a populated section, which must end with `.`. +Do not use the Unicode em dash character `—` in authored release-note prose, including the opening summary, release-highlight bullets, alert prose, or any generated explanatory text around the sources. Prefer natural sentence continuation instead of definition-style punctuation after a bold lead-in. + +The exact `Sources:` entries are an evidence-preservation surface, not rewritten prose. If a verified PR title or commit subject already contains `—`, preserve that source line exactly rather than normalizing the title and breaking source verification. ### GitHub alert blocks (optional) @@ -271,6 +287,8 @@ Alert blocks appear after the summary and before the `Sources:` section. Do not invent alerts. Do not add a `WARNING` or `CAUTION` unless the release data supports that level of attention. Breaking changes should normally use `WARNING`. Security-sensitive or risk-heavy changes should normally use `CAUTION`. +Only supported GitHub alert blocks may appear here. Do not use ordinary blockquotes, ad hoc `>` callouts, or unsupported markers such as `> [!OTHER]`. + ### The Sources section The `Sources:` section preserves the original references that informed the summary. This gives readers a path to the raw details if the summary is not enough. @@ -324,14 +342,20 @@ Nothing may appear after this line. ## Non-Negotiable Rules - The first line of the output is exactly `## What's Changed`. +- The first non-empty summary line after that heading begins exactly with `This release `. +- The opening summary is exactly one concise paragraph. +- The opening summary paragraph is followed by one or more release-highlight bullets before any optional alert blocks or the `Sources:` section. +- Release-highlight bullets use bold lead-ins with natural sentence continuation; bold-leading prose paragraphs and `**** —` / `****: ` fragments are not allowed. - The summary covers all meaningful changes in the comparison range. - The summary is optimized for GitHub release notes, not raw commit history. - Alert blocks are included only when they add value and are supported by the release data. +- Only supported GitHub alert blocks appear after the release highlights; plain blockquotes and unsupported markers are not used there. - Alert severity matches the actual impact of the change. - The `Sources:` section is always included. - Source entries use the `* by <contributors> in <url>` format. Include all verified source authors with exact `@login` values, falling back to recorded names when a GitHub login is unavailable. - The final line is the full changelog link in the exact format shown above. - Nothing appears after the full changelog link. +- No Unicode em dash appears in authored release-note prose; exact source titles remain verbatim evidence. - No unsupported claims are invented. - Breaking changes, if any, are clearly identified. - Vague wording like "various improvements" or "miscellaneous changes" is avoided. @@ -390,22 +414,28 @@ For each commit in the compare range, check whether it belongs to a pull request Before returning the result, verify: 1. The first line is exactly `## What's Changed`. -2. The summary is human-friendly and optimized for GitHub release notes. -3. The summary covers the meaningful changes in the comparison range. -4. GitHub alert blocks are included only when they add value. -5. Alert severity matches the actual impact of the change. -6. Alert blocks are supported by the release data. -7. A `Sources:` section is included with all contributing PRs and commits. -8. Source entries use the `* <title> by @<author> in <url>` format, with fallback to author name when no GitHub username is available. -9. All contributors in the comparison range are represented in the Sources section. -10. The final line is the full changelog link. -11. Nothing appears after the full changelog link. -12. No unsupported claims were invented. -13. Breaking changes, if any, are clearly identified. -14. When using default resolution, the comparison range correctly reflects the current branch against the upstream default branch. -15. Every commit has a completed association lookup; unresolved or capped data is explicitly marked incomplete. -16. Every qualifying PR appears once, no covered commit is also listed, and no unrelated or unmerged PR is included. -17. PR titles, URLs, and author logins match REST metadata exactly, including `[bot]`; no normalized app slug replaces a bot login. -18. Each qualifying PR's original commit inventory is complete, and every verified author/co-author is credited on its source line, even for squash merges and automated release PRs. -19. The summary covers meaningful final changes across all authors and changed files; stale PR descriptions do not override diff evidence. -20. Each prose paragraph and Markdown list item occupies one physical line, regardless of length. +2. The first non-empty summary line after that heading begins exactly with `This release `. +3. The opening summary is exactly one concise paragraph on one physical line. +4. At least one `- ` release-highlight bullet appears after the opening paragraph and before any optional alert blocks or the `Sources:` section. +5. Release-highlight bullets use bold lead-ins with natural sentence continuation instead of bold-label fragments such as `**<lead>** —` or `**<lead>**:`. +6. Bold-leading prose paragraphs are not used as a substitute for release-highlight bullets. +7. No Unicode em dash `—` appears in authored release-note prose; exact source titles remain verbatim evidence. +8. The summary is human-friendly and optimized for GitHub release notes. +9. The summary covers the meaningful changes in the comparison range. +10. GitHub alert blocks are included only when they add value. +11. Alert severity matches the actual impact of the change. +12. Alert blocks are supported by the release data and use only the supported GitHub alert markers. +13. A `Sources:` section is included with all contributing PRs and commits. +14. Source entries use the `* <title> by @<author> in <url>` format, with fallback to author name when no GitHub username is available. +15. All contributors in the comparison range are represented in the Sources section. +16. The final line is the full changelog link. +17. Nothing appears after the full changelog link. +18. No unsupported claims were invented. +19. Breaking changes, if any, are clearly identified. +20. When using default resolution, the comparison range correctly reflects the current branch against the upstream default branch. +21. Every commit has a completed association lookup; unresolved or capped data is explicitly marked incomplete. +22. Every qualifying PR appears once, no covered commit is also listed, and no unrelated or unmerged PR is included. +23. PR titles, URLs, and author logins match REST metadata exactly, including `[bot]`; no normalized app slug replaces a bot login. +24. Each qualifying PR's original commit inventory is complete, and every verified author/co-author is credited on its source line, even for squash merges and automated release PRs. +25. The summary covers meaningful final changes across all authors and changed files; stale PR descriptions do not override diff evidence. +26. Each prose paragraph and Markdown list item occupies one physical line, regardless of length. diff --git a/skills/git-remote-release/evals/evals.json b/skills/git-remote-release/evals/evals.json index 115a456..b6e508f 100644 --- a/skills/git-remote-release/evals/evals.json +++ b/skills/git-remote-release/evals/evals.json @@ -17,10 +17,14 @@ { "id": 1, "prompt": "Generate release notes for https://github.com/codebeltnet/agentic/compare/v0.4.5...v0.4.6", - "expected_output": "Release notes starting with ## What's Changed, a human-friendly summary of changes between v0.4.5 and v0.4.6, a Sources section with PR/commit references, and ending with the full changelog link.", + "expected_output": "Release notes starting with ## What's Changed, a concise `This release ...` opening paragraph, curated dash bullets covering the important outcomes between v0.4.5 and v0.4.6, a Sources section with PR/commit references, and ending with the full changelog link.", "expectations": [ "First line is exactly ## What's Changed", + "First non-empty summary line after the heading begins with This release ", "Parses the compare URL to extract owner, repo, previous tag, and current tag", + "Opening summary is one concise release-level paragraph", + "At least one curated dash bullet follows the opening paragraph before Sources", + "Release-highlight bullets use bold lead-ins with natural sentence continuation", "Summary explains the effect of changes rather than listing raw commit messages", "Sources section includes all contributing PRs and/or commits with author and URL", "Final line is the full changelog compare link in the exact required format", @@ -118,6 +122,24 @@ "Does not claim unverified runtime signing improvements or merely repeat the dependency-only PR body", "Ends with the v10.0.11...v10.0.12 compare link without duplicate covered commit sources" ] + }, + { + "id": 10, + "prompt": "Generate release notes for codebeltnet/aws-signature-v4 v10.0.11...v10.0.12 using only the attached GitHub API snapshot in aws-signature-v4-v10.0.12.json. A previous attempt produced `## What's Changed` followed by bold-leading prose paragraphs such as `**Dependency and tooling updates** — Updated dependencies ...`. Reject that presentation and produce the required paste-ready release story instead.", + "files": [ + "evals/files/aws-signature-v4-v10.0.12.json" + ], + "expected_output": "Begins the optimized summary with a concise `This release ...` paragraph, follows it with curated dash bullets using bold lead-ins and natural prose, groups the shipped outcomes instead of reproducing the evidence inventory, contains no em dash, preserves the verified contributor-complete PR #34 source line, and ends with the compare link.", + "expectations": [ + "First non-empty summary line after the heading begins with This release ", + "Opening summary is one concise release-level paragraph", + "At least one curated dash bullet appears before Sources", + "Release-highlight bullets use bold lead-ins with natural sentence continuation instead of bold-leading paragraphs", + "Does not use the Unicode em dash character anywhere in the returned release notes", + "Groups dependency upgrades, repository standards, test-platform migration, coverage migration, and contributing documentation into release outcomes instead of a source-by-source inventory", + "Preserves the single PR #34 source line crediting @codebelt-aicia[bot], @aicia-bot and @gimlichael", + "Ends with the exact v10.0.11...v10.0.12 compare link" + ] } ] } diff --git a/skills/git-remote-release/scripts/collect-release-evidence.py b/skills/git-remote-release/scripts/collect-release-evidence.py index 0724d4f..c941694 100644 --- a/skills/git-remote-release/scripts/collect-release-evidence.py +++ b/skills/git-remote-release/scripts/collect-release-evidence.py @@ -47,6 +47,107 @@ def source(title, url, authors): return f"* {title.splitlines()[0]} by {credit} in {url}" +SUPPORTED_ALERT_START = re.compile(r"^> \[!(NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]$") +ANY_ALERT_MARKER = re.compile(r"^> \[!([A-Z]+)\]") + + +def verify_summary(lines): + errors = [] + source_index = lines.index("Sources:") + section = lines[1:source_index] + content_positions = [index for index, line in enumerate(section) if line.strip()] + + if not content_positions: + return ["Missing release summary before Sources"] + + opening_index = content_positions[0] + opening = section[opening_index] + if not opening.startswith("This release "): + if opening.startswith("**"): + errors.append("Bold-leading prose paragraphs are not allowed; start with 'This release ' and use dash bullets for release highlights") + else: + errors.append("First non-empty summary line must begin with 'This release '") + + remaining = content_positions[1:] + if not remaining: + errors.append("Release summary must include at least one dash bullet after the opening paragraph") + return errors + + first_after_opening = remaining[0] + if section[first_after_opening].startswith("**"): + errors.append("Bold-leading prose paragraphs are not allowed; use dash bullets for release highlights") + if not section[first_after_opening].startswith("- "): + errors.append("Opening summary paragraph must be followed by release-highlight bullets") + + bullets = [] + within_alert_block = False + alert_has_content = False + for line in section[first_after_opening:]: + if not line.strip(): + continue + if line.startswith("- "): + if within_alert_block: + errors.append("Release-highlight bullets must appear before any alert blocks") + else: + bullets.append(line.rstrip()) + continue + + if SUPPORTED_ALERT_START.fullmatch(line): + if within_alert_block and not alert_has_content: + errors.append("GitHub alert blocks must be followed by at least one content line") + within_alert_block = True + alert_has_content = False + continue + + if line.startswith(">"): + marker = ANY_ALERT_MARKER.match(line) + if within_alert_block: + if marker: + errors.append("GitHub alert blocks must use only supported alert markers on standalone marker lines") + elif line[1:].strip(): + alert_has_content = True + continue + if marker: + errors.append("GitHub alert blocks must use only supported alert markers on standalone marker lines") + else: + errors.append("Plain blockquotes are not allowed after the release highlights; use supported GitHub alert blocks only") + continue + + within_alert_block = False + if line.startswith("**"): + errors.append("Bold-leading prose paragraphs are not allowed; use dash bullets for release highlights") + else: + errors.append("Only release-highlight bullets and supported GitHub alert blocks may follow the opening paragraph") + + if within_alert_block and not alert_has_content: + errors.append("GitHub alert blocks must be followed by at least one content line") + + if not bullets: + errors.append("At least one release-highlight bullet is required before Sources") + return errors + + for bullet in bullets: + if re.match(r"^- \*\*[^*]+\*\*\s*[:—-]", bullet): + errors.append("Release-highlight bullets must continue naturally after the bold lead-in, not with label punctuation") + break + match = re.match(r"^- \*\*[^*]+\*\*\s+(.+)$", bullet) + if not match: + errors.append("Release-highlight bullets must use a bold lead-in followed by natural sentence prose") + break + if not re.search(r"[\w`]", match.group(1)): + errors.append("Release-highlight bullets must include explanatory prose after the bold lead-in") + break + + for bullet in bullets[:-1]: + if not bullet.endswith(","): + errors.append("Each non-final release-highlight bullet must end with a comma") + break + if not bullets[-1].endswith("."): + errors.append("The final release-highlight bullet must end with a period") + + return errors + + def collect(repository, previous, current, api=github): if not re.fullmatch(r"[\w.-]+/[\w.-]+", repository): raise ValueError("Repository must be owner/repo") @@ -165,11 +266,18 @@ def verify(evidence, draft): if not evidence["complete"]: errors.append("Evidence is incomplete: " + "; ".join(evidence["issues"])) lines = draft.strip().splitlines() + if lines.count("Sources:") == 1: + authored_lines = lines[:lines.index("Sources:")] + else: + authored_lines = lines + if any("—" in line for line in authored_lines): + errors.append("Unicode em dash is not allowed in authored release-note prose") if not lines or lines[0] != "## What's Changed": errors.append("Missing required opening heading") if lines.count("Sources:") != 1: errors.append("Expected exactly one Sources section") else: + errors.extend(verify_summary(lines)) actual = [line for line in lines[lines.index("Sources:") + 1:-1] if line.strip()] if actual != evidence["sources"]: errors.append("Sources must match collected source lines, including all contributors, exactly") diff --git a/skills/git-remote-release/scripts/test-release-evidence.py b/skills/git-remote-release/scripts/test-release-evidence.py index 053c4a0..e3b60a2 100644 --- a/skills/git-remote-release/scripts/test-release-evidence.py +++ b/skills/git-remote-release/scripts/test-release-evidence.py @@ -46,8 +46,13 @@ def api(self, endpoint): def collect(self): return collector.collect("example/widget", "v1", "v2", self.api) - def draft(self, evidence): - return "## What's Changed\n\nUpdated contributor workflow.\n\nSources:\n\n" + "\n".join(evidence["sources"]) + "\n\n**Full Changelog**: https://github.com/example/widget/compare/v1...v2" + def draft(self, evidence, summary=None): + summary = summary or ( + "This release improves contributor-complete verification for automated service updates.\n\n" + "- **Contributor attribution** now keeps every verified PR contributor on the shared source line,\n" + "- **Draft verification** rejects incomplete sources and malformed release-summary structure." + ) + return "## What's Changed\n\n" + summary + "\n\nSources:\n\n" + "\n".join(evidence["sources"]) + "\n\n**Full Changelog**: https://github.com/example/widget/compare/v1...v2" def test_squash_expands_all_pages_and_preserves_changes(self): evidence = self.collect() @@ -64,6 +69,86 @@ def test_missing_contributor_and_duplicate_source_fail_verification(self): self.assertTrue(collector.verify(evidence, draft.replace("Sources:", "Sources:\n" + evidence["sources"][0]))) self.assertTrue(collector.verify(evidence, draft.replace("Sources:", "Sources:\n- Extra source by @someone"))) + def test_source_titles_with_em_dash_still_verify(self): + self.pr["title"] = "Service — update" + evidence = self.collect() + self.assertTrue(evidence["complete"], evidence["issues"]) + self.assertIn("—", evidence["sources"][0]) + self.assertEqual(collector.verify(evidence, self.draft(evidence)), []) + + def test_summary_structure_rejects_bold_paragraph_regression(self): + evidence = self.collect() + draft = self.draft(evidence, ( + "**Team catalog matching precision** — The API now matches exact team components.\n\n" + "**Enhanced test coverage and infrastructure** — Comprehensive test fixtures exercise overlapping names.\n\n" + "**Dependency and tooling updates** — Updated dependencies and repository tooling." + )) + errors = collector.verify(evidence, draft) + self.assertTrue(any("This release " in error for error in errors), errors) + self.assertTrue(any("Bold-leading prose paragraphs" in error for error in errors), errors) + self.assertTrue(any("dash bullet" in error for error in errors), errors) + self.assertTrue(any("Unicode em dash" in error for error in errors), errors) + + def test_summary_structure_requires_bullets_and_natural_bold_leads(self): + evidence = self.collect() + without_bullets = self.draft(evidence, "This release improves contributor-complete verification.") + self.assertTrue(any("dash bullet" in error for error in collector.verify(evidence, without_bullets))) + + label_style = self.draft(evidence, ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution**: now keeps every verified PR contributor on the shared source line." + )) + self.assertTrue(any("label punctuation" in error for error in collector.verify(evidence, label_style))) + + def test_summary_structure_rejects_plain_blockquotes_and_unsupported_alert_markers(self): + evidence = self.collect() + for summary, expected in ( + ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution** now keeps every verified PR contributor on the shared source line.\n\n" + "> Just a blockquote.", + "Plain blockquotes are not allowed", + ), + ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution** now keeps every verified PR contributor on the shared source line.\n\n" + "> [!OTHER]\n" + "> Unsupported marker.", + "supported alert markers on standalone marker lines", + ), + ): + with self.subTest(expected=expected): + errors = collector.verify(evidence, self.draft(evidence, summary)) + self.assertTrue(any(expected in error for error in errors), errors) + + def test_summary_structure_requires_explanatory_prose_after_bold_lead(self): + evidence = self.collect() + summary = ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution** ." + ) + errors = collector.verify(evidence, self.draft(evidence, summary)) + self.assertTrue(any("explanatory prose" in error for error in errors), errors) + + def test_summary_structure_requires_bullet_punctuation(self): + evidence = self.collect() + for summary, expected in ( + ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution** now keeps every verified PR contributor on the shared source line.\n" + "- **Draft verification** rejects incomplete sources and malformed release-summary structure.", + "Each non-final release-highlight bullet must end with a comma", + ), + ( + "This release improves contributor-complete verification.\n\n" + "- **Contributor attribution** now keeps every verified PR contributor on the shared source line,", + "The final release-highlight bullet must end with a period", + ), + ): + with self.subTest(expected=expected): + errors = collector.verify(evidence, self.draft(evidence, summary)) + self.assertTrue(any(expected in error for error in errors), errors) + def test_failed_association_is_not_a_direct_commit(self): self.routes["/commits/squash/pulls?per_page=100"] = RuntimeError("HTTP 403") evidence = self.collect()