setup: guard the site base path, run the dormant quality gates, pin the toolchain - #23
Merged
Conversation
Content links are written root-relative in Markdown and get the `/caseum`
deploy base prefixed at render time by the `rehypeBaseLinks` plugin in
`docs/astro.config.mjs`. That plugin is wired in through
`markdown.rehypePlugins`, which Astro 7 now reports as deprecated in favour of
passing plugins to `unified({...})` from `@astrojs/markdown-remark`.
When that option is eventually removed the plugin simply stops being called.
Verified by removing it and rebuilding: the build printed "Complete!", exit 0,
and `astro check` still reported zero errors, while 117 in-content links and
image sources lost their prefix and would have 404ed on the published site.
There is no natural alarm for that, so this adds one.
The check greps the built HTML rather than unit-testing the plugin function,
because a unit test would keep passing while Astro stopped calling it.
Assisted-by: Claude:claude-opus-5
Co-Authored-By: lsimons-bot <bot@leosimons.com>
Tools were pinned to minor versions (`bun = "1.3"`), which resolves to whatever is newest at install time. Exact-pin them so an upstream compromise cannot arrive by auto-download on a laptop or a runner, and so a bump is a reviewable commit. Adds zizmor, actionlint and shellcheck as pinned local tools. zizmor is pinned to 1.29.0, the same version `.github/workflows/ci.yml` asks the zizmor action for, so the local and CI audits cannot silently diverge. shellcheck is load-bearing rather than decorative: actionlint exits 0 without ever inspecting a `run:` block when shellcheck is absent. New tasks: - `lint` - the prek hooks over all files - `lint-links`- the lychee link check, on its own - `audit` - zizmor then actionlint - `docs-verify` - the base-path guard added in the previous commit - `ci` - all of the above in CI's order `lint` skips lychee and `lint-links` runs it separately for two measured reasons. prek shards files across parallel hook invocations, and eight concurrent lychee processes hammer the same hosts hard enough to fail on a URL that returns 200 when checked alone - observed on www.ietf.org. And a naive `**/*.md` glob also matches the ~520 dependency READMEs under `docs/node_modules`, turning a 4-second check of 33 tracked files into a ten-minute crawl of other people's links; `lint-links` drives lychee from `git ls-files` instead. `ci` is a list of explicit `mise run` calls rather than a `depends` list because mise may run `depends` entries in parallel and these are strictly ordered. Quarto gains a `docs-presentation` task with a task-scoped tool pin rather than a `[tools]` entry: it is a large download, the rendered deck is committed, and the source changes about once a year. Verified that a task-scoped tool is absent from `mise ls --current` and is not fetched by `mise install`, so neither CI nor a new contributor pays for it. prek gains a shellcheck hook, since `mise run audit` only reaches shell inside workflow `run:` blocks and not standalone scripts. Verified it fails on a deliberately introduced SC2154. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
`prek.toml` defines formatting, Markdown lint, secret detection and link checking, but nothing ran them outside a contributor's own machine, and only then if they had run `prek install`. The hooks are not installed in this repository's clone at all, so in practice they have been gating nothing. Anything arriving by web edit, from dependabot, or from a fresh clone bypassed them entirely. Adds two jobs: - `lint` - `mise run lint` plus `mise run audit` (zizmor and actionlint, with shellcheck present so `run:` blocks are actually inspected), with the prek hook environments cached the same way the bun cache already is. - `links` - `mise run lint-links`, kept separate so that when it goes red it is unambiguous that the cause is a third-party host and not the build. `deploy.yml` does not depend on it, so a link outage cannot block a deploy. Pins the zizmor action's `version` input to 1.29.0, matching the pinned local binary. Confirmed 1.29.0 is present in the version manifest baked into zizmor-action v0.6.2 (`support/versions` at the pinned SHA), and that the pinned SHA is a commit object rather than an annotated tag; the input is validated against that baked-in list, so an unsupported value fails the job before zizmor runs. Both `ci.yml` and `deploy.yml` also gain the `docs-verify` step. Blocking the deploy on it is deliberate: publishing a site whose in-content links have all lost the base path is the failure worth preventing. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
The "Autogenerated files" block ignored `docs/index.md`, `docs/CODE_OF_CONDUCT.md`, `docs/CONTRIBUTING.md`, `docs/LICENSE.txt` and `site/`. Those were MkDocs outputs; `mkdocs.yml` and `site/` were deleted in c7593e5, "build(docs): rebuild site with Astro Starlight". The Astro build writes to `docs/dist`, which `docs/.gitignore` already covers. Keeping them is not harmless. `docs/` is now the site root, so a real `docs/index.md` would be silently untracked, and `site/` was unanchored and would have swallowed a directory of that name at any depth. `out/` had the same shape and is now root-anchored. Adds `.claude/worktrees`, matching the other repositories. `.claude/` itself stays tracked on purpose. Note for anyone with an existing clone: four stale MkDocs-era files may still be sitting in `docs/` and will now show up as untracked. They are pre-Astro leftovers and can be deleted. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
… agents Adds `docs/agents/issue-tracker.md`: the tracker in use, the triage workflow, and the full label set including the four triage labels created on the remote during this pass (needs-triage, needs-info, ready-for-agent, ready-for-human). It records honestly that the Issues tab is currently disabled on this repository, with the command to turn it on, rather than describing a tracker that would fail on first use. The file sits outside `docs/src/content/docs/`, so Astro does not publish it. `AGENTS.md` gains the agent-skills section (git remote, issue tracker, triage labels), a task list that matches `mise tasks` again - it was missing docs-preview, docs-clean, docs-favicon and ci-watch as well as everything added in this branch - and a note on the deprecated `markdown.rehypePlugins` hook and the guard that now protects it. `CONTRIBUTING.md` was missing the `prek install` step entirely, described `quarto render` as a bare command with no pinned version, and listed three of the available tasks. It now also points security reports at private vulnerability reporting, which was enabled on the repository during this pass. `docs/src/content/docs/contributing.md` is a hand-kept copy of that text with site-relative links; both are updated, and the root file now says so, because nothing enforces that they stay in step. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
Applies an independent review of this branch. The issue tracker docs asserted the Issues tab was disabled. It is enabled. The reading was correct when it was made and the setting was changed afterwards, which is precisely the problem: `docs/agents/issue-tracker.md` exists to tell an agent whether it can file issues, and it quoted both a command and its expected output, so a reader was actively discouraged from running the command that would have shown the doc to be wrong. The claim is deleted rather than inverted - a mutable repository setting has no business being a committed fact. Same for the matching sentence in `AGENTS.md`. Also deleted: a sentence claiming caseum and lsimons.github.io were the only repositories in the fleet with issues disabled. Both halves were false (lsimons.github.io has issues enabled, and 29 other repositories do not). It was reasoning about *other* repositories committed where nothing in this repository could ever falsify it. A leaf repo's agent docs should make no claims about other repositories at all. `AGENTS.md` said `deploy.yml` "deliberately runs only the build" - while this same branch adds the base-path check to the deploy job. The intent (the link check does not gate deploys) was right, but as written it denied this branch's most consequential change to the production path. Reworded, and the deliberate double zizmor run is now documented so nobody tidies away the wrong copy. Drops `wontfix` from the triage-label list: the standing set is four, and `wontfix` is a pre-existing GitHub default that is a closing label, not a triage label. Adds the shellcheck hook to the hook list, and records that commitlint is a `commit-msg` hook and therefore gated locally only, never in CI. The base-path guard had two blind spots, both found by testing it rather than reading it: - The section list was hardcoded, so a content section added later would have been silently unguarded - the one way this check could get quieter over time instead of louder. It is now derived from `src/content/docs/` at runtime. Verified by adding a section: the guard picked it up unprompted and reported 9 sections, then returned to 8 when it was removed. - `--include=index.html` skipped `404.html`, the one generated page the filter excluded. Now included; verified by injecting a bare link into `404.html` and watching the guard fail on it. The guard is also now invoked as `bash <script>` rather than `./<script>`. It gates a production deploy, and an executable bit is easy to lose to a web edit or a `core.fileMode=false` clone. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
Standing fleet decision: the license file is `LICENSE`, no extension. I had left this alone on the grounds that a rename "touches five places". Review showed that was wrong, and I re-checked it rather than taking the correction on trust: only `README.md` and `CONTRIBUTING.md` link to the root file. The Starlight sidebar entry, `docs/src/content/docs/index.md` and `docs/src/content/docs/contributing.md` all use the site-absolute `/LICENSE.txt`, which resolves to `docs/public/LICENSE.txt` - a separate, byte-identical tracked copy that backs the published URL. `AGENTS.md` likewise describes that copy, not this one. So this is a `git mv` and two link edits. `docs/public/LICENSE.txt` is deliberately untouched, and `https://lsimons.github.io/caseum/LICENSE.txt` is unaffected: verified by rebuilding and confirming `docs/dist/LICENSE.txt` is still produced and still byte-identical to the root file. The licence text itself is unchanged - this is CC-BY-4.0 either way, and the content of the file was not read, edited or reconsidered. GitHub's licence detection reads `LICENSE` and `LICENSE.txt` equally well, so nothing is lost. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
Before this, a broken internal link was caught by nothing. Measured, not argued - adding `[nope](/guides/this-page-does-not-exist/)` to a content page gave: astro build "Complete!", exit 0 astro check 0 errors verify-base-path.sh "OK - 876 base-prefixed content links" lychee "0 Errors" Every one of those is behaving as designed: - Astro 7 does not validate Markdown links, and `starlight-links-validator` is not installed. Starlight validates sidebar `slug:` entries only. - `verify-base-path.sh` checks that the `/caseum` prefix was applied. A link to a page that does not exist still gets prefixed, so the broken link counted as one of the 876 successes. - `.lychee.toml` resolves root-relative links against `base_url` and then excludes that domain, so lychee never requests an internal link. The links most likely to break from a change in this repository - its own - were the only ones nothing checked, while four links to a third-party glossary were blocking a merge. This inverts that. `verify-internal-links.sh` resolves every internal link in the built output against the built output, handling directory routes, extensionless pages and files, and skipping Astro's own hashed assets. It reads the build rather than the Markdown source on purpose: what matters is whether the published URL resolves after base prefixing, redirects and Starlight's route generation have had their say. It also covers the hand-written redirect stubs from `public/`, where a stale target is exactly as broken as a stale content link. Verified both ways: 64 internal links resolve on the current tree with no false positives, and the probe link above is caught (`1 of 65 ... point at nothing`). Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
PR #23's `Link check (lychee)` job failed with **zero errors** and four timeouts, all to `agilealliance.org`, on a branch that touched none of those links. Retrying was considered first and ruled out on evidence rather than taste: lychee already defaults to `--max-retries 3` with a 20s timeout, so the failure had *already survived three retries per link*. Raising the timeout only helps a host that is slow, and this one is not - both probed URLs return 301 in ~1.2s from a normal client. A host that answers instantly from anywhere except a CI runner is blocking by IP, and no retry or timeout setting fixes that. So the job was gating merges on a third party's rate limiter, for a class of failure the pull request could not cause and the author could not fix. Any external host can do this on any PR, and the next one will not be `agilealliance.org`. External link checking moves to `links.yml`, weekly and on demand. **What is no longer gated:** reachability of third-party URLs. Rot there is real, but it is not caused by the pull request in front of you and does not need fixing inside its lifetime. **What is still gated on every PR:** `mise run docs-verify` in the build job - the base-path check and the new internal-link check. Both are offline and deterministic, and between them they cover every link this repository controls. Link coverage on a PR is strictly better than before this branch, and better in the half that this repository can actually break. `agilealliance.org` is also excluded in `.lychee.toml`, in the same style and for the same reason as the existing ESA host: a timeout that happens only from CI is not a broken link, and leaving it in would make the weekly job permanently red - which is worse than useless, because it trains everyone to ignore it. Verified the exclusion is surgical: excluded links go 182 -> 186, the four in question, with OK 112 -> 108 and still 0 errors. Also corrects a false comment in `.lychee.toml` claiming "the Astro build and the build-time link audit validate them for real" of internal links. They did not; see the previous commit. The comment now names what actually does it. Assisted-by: Claude:claude-opus-5 Co-Authored-By: lsimons-bot <bot@leosimons.com>
Owner
Author
Update: the merge gate is now three jobs, not fourThe first PR run failed on Raising the timeout was ruled out on evidence rather than taste: lychee 0.24.2 defaults to Two commits followed:
Net effect: PR link coverage is better than before this branch, in the half a change to this repo can actually break. What is no longer gated per-PR is third-party reachability, which this repo cannot control and which was the thing failing. Not verifiable from here: that the lychee job stops failing — a runner-IP block can't be reproduced locally. It no longer runs on PRs either way. Note Co-Authored-By: lsimons-bot bot@leosimons.com |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fleet-wide
/setuprun, wave 3 batch 2. A1 → A2 (independent) → A1 chain; 7 commits.The headline: a silent, total link breakage on the published site
markdown.rehypePlugins— deprecated in Astro 7 — is howrehypeBaseLinksprefixes/caseumonto every content link. Removing it produces a build that looks entirely healthy:…while 117 links silently lose their prefix, every one a 404 on the published site. Nothing in the toolchain notices.
This PR adds
docs/scripts/verify-base-path.sh, wired into both CI and the deploy job. The reviewer attacked the guard across eight builds rather than accepting the one reproduction, and it fails closed on: an emptydist, a zero-link build, a total regression, a partial regression (900 good links + 1 bad), and a renamed output directory. Two blind spots found that way are also fixed — the section list is now derived fromsrc/content/docs/at runtime instead of hardcoded (so a new content section is guarded automatically), and404.htmlis now scanned.The quality gates existed; nothing ran them
prek.tomldefined mdformat, markdownlint, lychee, gitleaks and commitlint — and no workflow invoked any of them..git/hooks/held only.samplefiles, soprek installhad never been run locally either. In practice they gated nothing: web edits, dependabot PRs and fresh clones all bypassed them. Now wired in aslintandlinksjobs.Commit messages remain hook-gated locally only — a CI commitlint step needs a PR commit-range strategy, and inventing one during this pass was out of scope.
AGENTS.mdnow says so plainly rather than implying coverage.Scaffolding
lint/lint-links/audit/docs-verify/citasks.version: 1.29.0, after confirming1.29.0is in the version manifest baked into zizmor-action v0.6.2 at the pinned SHA — the input is validated against a list inside the action's own release, so the two are coupled..gitignorerules and root-anchored the unanchoredsite//out/patterns. Exactly four dead files are un-ignored; nothing live was hidden.LICENSE.txt→LICENSE(§4.7). Only two files linked to the root copy; the sidebar and content links resolve todocs/public/LICENSE.txt, a separate copy backing the published URL, which is unaffected — verified by rebuilding.docs/agents/issue-tracker.md; refreshedAGENTS.md,CONTRIBUTING.md,README.md.Repo settings
Private vulnerability reporting is now enabled —
caseumwas one of only two public repos in the fleet without it. Dependabot security updates flipped on (alerts were already on),sha_pinning_requiredwas alreadytrue, and the four#e6e6fatriage labels were created. All GET-verified.What review caught
Four false statements in the new agent-facing docs, all prose, all in the files whose entire job is to be trusted by an agent:
has_issuesclaim quoting the command and its expected output, which pre-empts the one check that would expose it as stale.AGENTS.mdstatingdeploy.yml"deliberately runs only the build" — denying this branch's own most consequential production-path change.Merge gate
deploy.ymlnow installs all seven tools on the production path, so wait for all four jobs green before merging. The risk is contained: the PR's own jobs perform the identical install, and all three new pins have Linux x86-64 release assets.Co-Authored-By: lsimons-bot bot@leosimons.com
Assisted-by: Claude:claude-opus-5