Skip to content

feat: hydrogen skills check for checking if skills are up to date - #3994

Open
fredericoo wants to merge 10 commits into
fb-skills-syncfrom
fb-skills-update
Open

feat: hydrogen skills check for checking if skills are up to date#3994
fredericoo wants to merge 10 commits into
fb-skills-syncfrom
fb-skills-update

Conversation

@fredericoo

@fredericoo fredericoo commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Stacked on #3992. Merge that first; this diff is the commits after edf93705a.

TL;DR: #3992 makes skills resyncable, but nothing fails when someone bumps Hydrogen and forgets to resync. This adds hydrogen skills check, a read-only command that exits non-zero when the synced skills do not match the installed package, so CI can enforce it.

Before

pnpm add @shopify/hydrogen@preview
# ...forget the next line and agents keep following last version's skills, silently
npx @shopify/hydrogen skills sync

After

npx @shopify/hydrogen skills check
# Hydrogen skills are out of date with @shopify/hydrogen 2026.10.0-preview.3 (5 to update, 2 new, 1 removed upstream). Run `npx @shopify/hydrogen skills sync`.
# exit 1
{
  "scripts": {
    "check:skills": "hydrogen skills check",
    "dev": "hydrogen skills check --mode=warn && vite dev"
  }
}

What this changes

  • hydrogen skills check (cli/skills.ts checkSkills): runs the same planner skills sync executes, in read-only mode, and throws with a one-line summary when a sync would add, update, or remove anything, when locally modified skills are behind, or when an unmanaged directory collides with a shipped name. A project that never synced fails too; running the command is the opt-in. Exit 0 prints the version it is aligned with. --mode=warn prints the same message and exits 0, for dev scripts that should nag but not block; --mode=error is the default.
  • getSkillsSyncStatus() / describeSkillsSyncStatus(): the read-only view and its message, split out so the CLI cannot disagree with what sync would do. Counts distinct skills rather than directories, since both .claude/skills and .agents/skills are written.
  • syncSkills takes force: boolean instead of raw argv; cli/index.ts parses flags once at the boundary.
  • hydrogen skills check --mode=warn && <dev> is the standard dev script. Both templates ship it, and the hydrogen-setup skill (scaffold step) instructs agents to prefix the app's existing dev command with it and verifies the chain in the final step. Warn mode prints nothing when skills are current, so the only output is the resync nudge after an upgrade.
  • Verified against the release path: prepare:preview-dist syncs skills into both harness roots, and the compiled template's dev prefix is silent afterwards.

Developer impact

Includes a minor changeset for @shopify/hydrogen: one new CLI subcommand under the existing skills namespace. setup and skills sync behaviour is unchanged. No new package exports.

Template source in this monorepo has no skills (they are synced only when compiling dist-preview), so running pnpm dev inside templates/* here prints the "19 new" nudge every time. Compiled templates are silent. Run pnpm exec hydrogen skills sync inside a template to quiet it locally; the generated .claude/skills and .agents/skills are untracked.

Out of scope

  • No hydrogen update command and no Vite dev-server warning. Both were built and dropped in review: a top-level command is a permanent compatibility surface for something package managers already do, and there is no umbrella Hydrogen Vite plugin to hang a warning on. skills check in CI is the single gate.
  • The hydrogen-setup skill does not yet tell agents to add skills check to a consumer's CI. Deliberate for now.

How to Test

  1. Run pnpm install && pnpm build:pkgs.
  2. In a scratch directory: mkdir app && cd app && echo '{"name":"app","dependencies":{"@shopify/hydrogen":"*"}}' > package.json.
  3. Run node <repo>/packages/hydrogen/bin/hydrogen.mjs skills check. Confirm it reports 19 new skills and exits 1.
  4. Run node <repo>/packages/hydrogen/bin/hydrogen.mjs skills sync, then skills check again. Confirm it reports up to date and exits 0.
  5. Edit version: in both .agents/skills/hydrogen-money/SKILL.md and .claude/skills/hydrogen-money/SKILL.md to an older version. Run skills check. Confirm it reports 1 to update and exits 1, and that no files changed.
  6. Append a line to both hydrogen-money/SKILL.md copies instead. Run skills check. Confirm it reports 1 locally modified and points at --force.
  7. Run skills check --mode=warn && echo continued. Confirm the message prints and continued follows. Run skills check --mode=loud and confirm it rejects the value.
  8. From the repo root run node scripts/preview-template-dist.ts prepare 2026.10.0-preview.2, then cd templates/react-router && pnpm exec hydrogen skills check --mode=warn. Confirm it prints nothing and exits 0. Restore with git restore templates and remove the generated .agents/.claude dirs.

@fredericoo fredericoo changed the title Add hydrogen update and a Vite skills drift warning Warn on dev startup when synced Hydrogen skills are out of date Sep 7, 2026
@fredericoo fredericoo changed the title Warn on dev startup when synced Hydrogen skills are out of date feat: warn on dev startup when synced Hydrogen skills are out of date Sep 7, 2026
@fredericoo fredericoo changed the title feat: warn on dev startup when synced Hydrogen skills are out of date Add hydrogen skills check for CI and dev scripts Sep 7, 2026
@fredericoo fredericoo changed the title Add hydrogen skills check for CI and dev scripts feat: hydrogen skills check for checking if skills are up to date Sep 7, 2026
update installs the current dist-tag with the detected package manager,
then runs skills sync. skillsSyncCheck() reuses the sync planner read-only
on dev-server startup and warns through the Vite logger when synced skills
no longer match the installed package. Package-manager detection moves out
of setup.ts so both commands share it.

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
…, take force as a boolean

- only skillsSyncCheck is exported from @Shopify/hydrogen/vite
- RunCommand/spawnRunCommand live in run-command.ts; gql, setup and
  update share it
- syncSkills and updateHydrogen take force; argv is parsed once in the
  CLI entry so bad flags fail before anything runs

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
- document the update forward-compat contract (installed CLI runs, tag
  moves must ship one release early, metadata must read across versions)
- resolve the project root upward from Vite root (Nuxt 4 style srcDir)
- warn once per process across multiple Vite servers
- spawn package managers through a shell on Windows (.cmd shims)

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
…eferences

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
A new CLI command is a permanent compatibility surface. Users bump the
dependency themselves and the dev-server warning tells them when to
resync. setup.ts and gql.ts return to their base-branch shape.

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
Shares the status description with the Vite warning; exits non-zero when
a sync would change anything, including when skills were never synced.

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
Removes skillsSyncCheck() and its template/example wiring, plus the
managed flag that only existed to keep the plugin quiet in projects that
never synced.

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
- templates prefix their dev scripts with it
- the hydrogen-setup skill instructs agents to do the same and verifies it
- validate:preview-dist runs checkSkills per template so dist-preview can
  never ship skills that drifted from the package they pin
- warn mode stays silent when skills are current

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
@fredericoo
fredericoo marked this pull request as ready for review September 7, 2026 16:38
@fredericoo
fredericoo requested a review from a team as a code owner September 7, 2026 16:38
"type": "module",
"scripts": {
"dev": "next dev",
"dev": "hydrogen skills check --mode=warn && next dev",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional, but good practice to keep them in check! this will always warn in local dev as expected, but when in dist-preview it'll work

Assisted-By: devx/d55d9e75-1efe-4d16-b7b7-20b3cae4124e
@fredericoo fredericoo self-assigned this Sep 8, 2026
@fredericoo fredericoo added the gsd:50917 New Hydrogen label Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gsd:50917 New Hydrogen

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant