A bounded website crawler for detecting broken links, missing assets, redirect loops, canonical URL problems, sitemap inconsistencies, orphan pages, and inaccessible downloads.
Release status: Initial public alpha. Bound crawl scope carefully and review findings before using them as deployment gates.
- Same-origin crawling with configurable origins, depth, page limits, concurrency, and exclusions
- Link, image, script, stylesheet, icon, manifest, preload, and download checks
- Redirect-chain and loop detection
- Missing and mismatched canonical URL detection
- Duplicate canonical detection
- Sitemap and sitemap-index availability and coverage checks
- Sitemap orphan-page detection
- Optional external-link checking
- JSON and Markdown reports with GitHub job summaries
- Docker image and GitHub Action
Requires Node.js 22.19 or newer.
npm install
npm run audit -- --config config.example.yamlReports are written to reports/content-integrity-report.json and
reports/content-integrity-report.md. Integrity failures return exit code 1; configuration and
execution errors return exit code 2.
Only URLs under allowed_origins are crawled as pages. Use maximum_pages, maximum_depth, and
exclude patterns to avoid logout routes, administrative areas, calendars, faceted search, or
other unbounded URL spaces.
See config.example.yaml and
schema/config.schema.json.
This repository is a generic crawler. It contains no private URLs, credentials, client content, internal topology, or CodeVelo production policy.
MIT © 2026 CodeVelo.dev LLC.