Skip to content

feat(llms): drop cookbook author pages and advertise .md URLs - #84

Merged
nibzard merged 1 commit into
mainfrom
feat/llms-index-hygiene
Jul 29, 2026
Merged

feat(llms): drop cookbook author pages and advertise .md URLs#84
nibzard merged 1 commit into
mainfrom
feat/llms-index-hygiene

Conversation

@nibzard

@nibzard nibzard commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Two index-hygiene changes for the agent-facing surfaces:

  • Drop cookbook author pages from the LLM index. Author pages are ego listings ("N recipes contributed by X") with little value for coding agents. They now carry `llm: false`, removing ~7 noise links from `llms.txt` and excluding them from `llms-full.txt`. `scripts/sync-cookbook.ts` emits the flag on author pages so the exclusion survives a re-sync. Topic hubs are kept: after the markdown-cleanup work (feat(llms): strip MDX noise from LLM-served markdown #83) they render as curated recipe link lists, which are useful agent navigation, so they stay in the index.
  • Update agent instructions to point agents at the `.md`-suffix convention (append `.md` to any docs URL) instead of the internal `/llms.mdx/` path, and to position `/llms-full.txt` as a large (~230k-token) corpus to fetch per-page from rather than read whole.

Notes

  • `sync-cookbook` regenerates all cookbook files and pulled in unrelated upstream recipe drift; I reverted that and re-applied only the author `llm: false` field, so this PR contains no unrelated content changes (and `cookbook.lock.json` is left untouched to stay consistent with the reverted content).
  • Dropped the planned "Overview Getting Started" title fix: only one page triggers the disambiguation and there's no safe one-line rule (using the page slug would turn "Sessions Api Quickstart" into "Quickstart Quickstart"). The entry has a clear description, so not worth the added generator complexity.

Testing

  • New `tests/cookbook-llm-flags.test.ts` guards the contract: every author page has `llm: false`, every topic page does not.
  • Regenerated `llms.txt`: 0 author entries, 18 topic entries retained; new agent-instruction lines present.
  • `bun test tests/` (201 pass), Biome (`--error-on-warnings`), `tsc --noEmit`, `bun run build` all clean.

Cookbook author pages are ego listings ("N recipes by X") with little
value for coding agents, so they now carry llm: false and are excluded
from the llms.txt index and llms-full.txt bundle. Topic hubs stay: after
the markdown-cleanup work they render as curated recipe link lists,
which are useful agent navigation. sync-cookbook now emits llm: false on
author pages so the exclusion survives a re-sync.

The agent instructions now point agents at the .md-suffix convention
(append .md to any docs URL) instead of the internal /llms.mdx/ path,
and position /llms-full.txt as a large corpus to fetch per-page from
rather than read whole.
@vercel

vercel Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 29, 2026 6:39pm

Request Review

@nibzard
nibzard merged commit ef4b1ca into main Jul 29, 2026
3 checks passed
@nibzard
nibzard deleted the feat/llms-index-hygiene branch July 29, 2026 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant