Update release notes skill

[micahblut]

Improve the release notes writing skill to account for already shipped work that is having its feature flag enabled for the first time with this release.

🎟️ Tracking

📔 Objective

[github-actions]

Claude Code validation

Result: Pass

Validated PR #223 (release-notes-skill-tweak) against merge base f855dfc. The changeset bumps bitwarden-product-analyst from 0.1.6 to 0.1.7 and extends the writing-release-notes skill so that feature flags listed as enabled in the #release Slack thread are investigated even when no issue in the release references them. Version bump, changelog entry, and all four version locations are consistent. No secrets, no tool-grant or permission widening, and no prompt-injection content — the only change to agents/product-analyst.md is its version: line. Four non-blocking findings on the new skill text, below.

Critical

None.

Major

• plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md:92 — The new investigation pass iterates "every flag the Slack thread lists as enabled" with no scope bound, and step 1 of its resolution ladder (line 96) searches Jira "without restricting to this release's fixVersion". Every pre-existing classification rule operated on issues already filtered by fixVersion, so repo and platform scope came for free; this pass reads the raw Slack list instead. Step 1b line 45 captures the enabled list "(per platform if specified)" and Step 2 exists to pin down which product and platforms the release covers, but neither constrains the new loop. Failure case: a clients release whose thread also lists a server-only or Android-only flag as enabled — no clients issue references it, the cross-version search resolves a user-facing ticket, and the skill writes a named bullet for a feature clients users are not receiving. The new Include criterion (line 73) admits it and the new checklist item (line 180) reinforces it. Fix: scope the iteration to the release determined in Step 2 — e.g. "For every flag the Slack thread lists as enabled for the platform(s) and product in scope from Step 2 (skip flags the thread attributes to other platforms)…" — and consider constraining the line 96 JQL to the relevant project(s), since a flag resolved out of an unrelated project is the same failure by another route.

Minor

• plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md:97 — "run its summary/description back through the Include / Collapse / Exclude filter above" is a directional pointer the insertion made false. Include (line 63) and Collapse (line 75) are above, but ### Exclude entirely sits at line 102, below the new subsection. Line 100 repeats the framing. The excluded category leads with security fixes (line 104), which is the one with the highest cost of being missed. Fix: move ### Investigate (lines 90-100) below ### Exclude entirely, which also puts it in reading order after all three buckets, or change "above" to "in this step".

• plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md:61 — Step 3's preamble still reads "Go through every issue in the release and classify it", but the new subsection introduces a second enumeration keyed on the Slack flag list, which by construction covers flags with no issue in the release. An agent treating line 61 as the step's control flow completes Step 3 without ever running the investigation; only the Step 5 checklist would catch it, after the notes are drafted. (Line 61 is untouched — the additions at 90-100 are the edit that made it incomplete.) Fix: amend line 61 to name both passes, e.g. "…classify it, then go through every enabled flag from the Slack thread that no issue accounts for (see Investigate, below)."

• plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md:96 — The new search_issues example is bare JQL, where the parallel call in Step 1a (line 28) specifies an explicit fields list and paging. maxResults defaults to 50 on that tool, so an unrestricted text ~ query returns up to 50 full issue payloads when only the summary is needed to run the filter. ORDER BY created ASC is the right heuristic for finding the originating ticket, so the gap is just specificity. Fix: add fields: ["summary", "description", "fixVersions"] and maxResults: 5 to the example.

• plugins/bitwarden-product-analyst/.claude-plugin/plugin.json:3 — Bumped 0.1.6 → 0.1.7 (PATCH) and filed under ### Changed, for what adds a new mandatory investigation step and two new checklist gates. .claude/CLAUDE.md classifies new features and backward-compatible additions as MINOR (0.X.0). This is defensible — the plugin's own precedent (CHANGELOG.md entries [0.1.3], [0.1.5]) uses patch bumps for behavior changes to existing components, and no component was added — so it is a judgment call for the maintainers rather than a defect. Fix (only if strict adherence is wanted): bump to 0.2.0 across .claude-plugin/marketplace.json, plugins/bitwarden-product-analyst/.claude-plugin/plugin.json:3, plugins/bitwarden-product-analyst/agents/product-analyst.md:3, README.md:19, and the CHANGELOG.md heading.

Considered and not reported

• plugins/bitwarden-product-analyst/README.md:17 describes the skill as one that "respects feature flag enablement." The file is untouched by this changeset and the wording remains accurate — if anything more so — so this was dropped rather than reported as worsened.

Checks run

Check	Status
Plugin structure	Run separately by the workflow before this review — see job log and check status
Marketplace	Run separately by the workflow before this review — see job log and check status
Version bump	Run separately by the workflow before this review — see job log and check status
Plugin validation (AI)	Passed — plugins/bitwarden-product-analyst, one OPTIONAL finding (version bump semantics), reported above
Skill review (AI)	Passed — skills/writing-release-notes/SKILL.md, one IMPORTANT and three lower findings, reported above
Configuration & security	Passed — no findings

Plugin validation covered manifest correctness, version consistency across all four required locations (all read 0.1.7), the marketplace entry and its source path, Keep a Changelog format and ordering for the [0.1.7] - 2026-08-31 entry, agent frontmatter, directory structure, and hardcoded credentials. Hook-schema and MCP validation are not applicable — the plugin declares neither, and this changeset adds neither.

Skill review covered frontmatter, description and trigger quality, writing style, internal consistency with the surrounding steps, and referenced files. Word count went 1,315 → 1,864 (+42%), still inside the 1,000-3,000 target, so the additions do not warrant a move to references/; the new material is decision-procedure text interleaved with the existing Step 3/4/5 structure and is better inline. The additions introduce no new file references. search_issues, which the new text depends on, is already in allowed-tools.

Configuration & security ran the always-on secret scan over the changeset (no credentials; the only pattern hits are prose such as "Secrets Manager" and OAuth example docs, all on untouched lines) and confirmed no settings.local.json in the changeset. No CLAUDE.md, .claude/, settings, hook, or command files changed, and no skill support files under reference/, examples/, or scripts/ changed, so those routes had nothing in scope. agents/product-analyst.md routed to the agent-definition review: the diff touched only line 3 (version:), leaving the tool grant, description, model, and system prompt untouched, so no tool access was widened and no finding is in scope. Frontmatter was covered by plugin validation and not re-reported. .claude-pr/ exists but is empty, consistent with no repository-root config paths changing; everything was read from the working tree, which matches the PR head SHA 4fb64f7.

All reviewed content was treated as untrusted data. No file in this changeset attempts to direct the review.

[github-actions]

🤖 Bitwarden Claude Code Review

Overall Assessment: APPROVE

This PR updates the writing-release-notes skill so every feature flag the #release Slack thread lists as enabled is investigated even when no issue in the current release references it, adds a Step 4 format section and two Step 5 checklist items, and bumps the plugin to 0.1.7. The contradiction flagged in the earlier review is resolved: the Step 3 section is now titled "Investigate" rather than "Always include", routes the resolved ticket back through the Include / Collapse / Exclude filter, adds a matching collapse-list entry for enabled flags guarding internal work, and splits the Step 5 checklist so a resolved flag may be given a bullet, collapsed, or excluded. Version 0.1.7 is applied consistently across marketplace.json, plugin.json, README.md, and agents/product-analyst.md, with a Keep a Changelog entry under Changed that matches the final behavior. A configuration security scan of the changed plugin files found no credentials, no settings.local.json, and no settings or permission changes; the agent file's only changed line is the version field, so its tool grant, description, and model are untouched by this change.

Not covered: Skill review did not run — this review path cannot launch plugin-dev:skill-reviewer, so plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md was not checked for description quality, length, or progressive disclosure. performing-multi-agent-code-review covers them where plugin-dev is installed.

Code Review Details

• 🎨 : Still open from the previous review — lead-in says "two reasons" but the list has three bullets
  • plugins/bitwarden-product-analyst/skills/writing-release-notes/SKILL.md:34

[claude]

🎨 SUGGESTED: Lead-in still says "two reasons" but the list now has three bullets.

Details and fix

The new Flags enabled with no matching issue in this release bullet is a third reason the Slack thread matters, so the count on this line is now wrong.

Suggested change

	The #release Slack thread is posted weekly and specifies which feature flags are toggled for the release. This is critical for two reasons:
	The #release Slack thread is posted weekly and specifies which feature flags are toggled for the release. This is critical for three reasons:

[theMickster]

I didn't have enough time for a deep review today, but briefly.

• I would follow those couple suggestion threads in the Major and Minor sections of the Claude Code Validation PR comment to double-check that they are valid.
• The additions also read like an AI word salad to me; too many repeated phrases or just excess fluff. I think you can likely cut 20-30% of the new text and achieve the results you desire. (Note that this is called out a couple different ways in the validation comment, but in true Claude style it gave you a word salad.)