Skip to content

SDK bundles ship generated excerpts of [arch]-cited architecture docs - #3914

Merged
borisbat merged 1 commit into
masterfrom
bbatkin/sdk-arch-extract
Aug 31, 2026
Merged

SDK bundles ship generated excerpts of [arch]-cited architecture docs#3914
borisbat merged 1 commit into
masterfrom
bbatkin/sdk-arch-extract

Conversation

@borisbat

Copy link
Copy Markdown
Collaborator

Installed SDK bundles carried dangling [arch] citations. The rule-document install ban keeps every ARCHITECTURE*.md out of the bundle, but shipped .das trees cite those documents. The smoke test hid the problem by running the daslib lint with LINT026 disabled, and the shipped MCP arch_of tool returned dead pointers for shipped code - the mechanism text that replaced comments was unreachable in an installed SDK.

The fix generates excerpts at build time. utils/internal/arch-extract resolves a tree's citations with the same scanner LINT026 uses and writes a banner-marked excerpt per cited document: the cited sections only, in document order, each anchor declared once (a cited anchor nested inside an emitted section rides its parent). Each shipping tree registers a "scan|base|dest" spec on the DAS_ARCH_EXTRACT_SPECS global property beside its own install rules - daslib, dasVulkan, dasMetal, dasSpirv, dasLLAMA - so a config that installs the citing tree also installs its excerpt, and a config that omits the tree installs neither. Both leak gates (the install-manifest check and the smoke-test find) admit only ARCHITECTURE*.md whose first line carries the generated banner. The smoke test now lints daslib with LINT026 armed and adds one closure lint per module excerpt: the proof that every shipped citation resolves and every shipped anchor is cited.

section_text moved from utils/mcp/tools/arch_common.das to utils/common/arch_citations.das (with a new section_span under it); the public re-export keeps call sites unchanged.

Where to look: the registry processor above the leak gate in the root CMakeLists.txt; emit_excerpt in utils/internal/arch-extract/main.das (the nested-anchor skip); the closure loop in ci/smoke_test_bundle.sh.

Validation, claims, ledger

Validation

  • Local cmake --install + ci/smoke_test_bundle.sh on a Ninja Release tree (dasLLAMA/dasMetal/dasSpirv on): all arch rows green - daslib lint with LINT026 armed, three per-module closure lints, both leak gates accepting the 11 excerpts. Three unrelated rows fail on this config as they always would (vulkan example/tutorial, Z3/VULKAN/VOLK licenses - dasVulkan and dasSMT not built locally); CI's bundle_smoke covers them.
  • The dasVulkan leg was simulated by hand (its daslib copied into the bundle, excerpt generated, bundle lint.exe run): 23 citations, 4 sections, 0 issues.
  • External codex round: 2 findings, both verified real, one fixed red-first (stale excerpt survives incremental re-extraction when a document's last citation disappears - the generated base tree is now rebuilt from nothing after validation, and CMake output dirs are keyed by base rather than list position), one scoped (below).
  • TDD audit with negative controls: 16 mutations named their failing test; three gaps it found are closed in this diff (vacuously-satisfiable document-order assertion, no nested-.das recursion fixture, unexercised root-containment guard).

Claims - stated, not tested

  • Cross-compiled bundles carry citations without excerpts (extraction needs a runnable host daslang). Deliberate: those bundles ship no lint.exe and never run the smoke test, so no gate regresses; behavior is unchanged from before this PR. A break would look like LINT026 findings from a target-built lint run over a cross-built SDK.
  • The generated-banner literal exists in three places (emitter, CMake gate, smoke gate); only the emitter side is test-pinned. Drift would surface as a bundle_smoke leak-gate failure, not silently.
  • Underscore-prefixed .das are skipped by the scan, mirroring LINT026's own walk; a _*.das that installs with a citation would dangle. No such file exists in any registered tree.

Not done

  • A backstop grep in the smoke test for arch(at= in bundle .das outside excerpt-covered folders (would catch a future module whose install rules land without a spec registration).

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 31, 2026 21:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes installed SDK bundles having dangling [arch] citations by generating and shipping banner-marked excerpts of the cited ARCHITECTURE*.md sections at build/install time, keeping LINT026 and MCP architecture navigation functional in the installed SDK.

Changes:

  • Add utils/internal/arch-extract tool + tests to generate excerpt ARCHITECTURE*.md files from [arch] citations.
  • Register shipping trees (daslib + select modules) via DAS_ARCH_EXTRACT_SPECS and install extracted excerpts alongside shipped code.
  • Re-arm LINT026 in bundle smoke tests and update leak gates to allow only banner-marked generated architecture excerpts.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
utils/mcp/tools/arch_common.das Removes local section_text implementation; relies on shared arch citation utilities.
utils/common/arch_citations.das Adds section_span and re-homes section_text for shared use by tools/lint/extractor.
utils/internal/arch-extract/main.das New extractor tool that scans shipped .das for citations and emits excerpt .md files.
utils/internal/arch-extract/tests/test_arch_extract.das New tests covering excerpt content/order/nesting, incremental cleanup, and error cases.
utils/CMakeLists.txt Adds arch-extract tests to the utils test target list.
CMakeLists.txt Adds arch-extract spec processing, build-time generation, installation, and leak-gate banner exception.
modules/dasVulkan/CMakeLists.txt Registers dasVulkan tree for excerpt extraction/installation.
modules/dasSpirv/CMakeLists.txt Registers dasSpirv tree for excerpt extraction/installation.
modules/dasMetal/CMakeLists.txt Registers dasMetal tree for excerpt extraction/installation.
modules/dasLLAMA/CMakeLists.txt Registers dasLLAMA tree for excerpt extraction/installation.
ci/smoke_test_bundle.sh Enables LINT026 on daslib in the bundle; adds per-module excerpt closure lint; allows generated excerpts through leak gate.
CLAUDE.md Documents the new “generated excerpts ship with SDK” rule for [arch]-cited architecture docs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread utils/internal/arch-extract/main.das
The rule-doc install ban left every shipped [arch] citation dangling in a
bundle: LINT026 fired on daslib (masked by --disable in the smoke test),
and the MCP arch_of tool returned dead pointers for shipped code.

utils/arch-extract resolves a tree's citations with the shared LINT026
scanner and emits banner-marked excerpts - the cited sections only, in
document order, a cited anchor nested inside an emitted section skipped
so the excerpt declares each anchor once. Shipping trees register on the
DAS_ARCH_EXTRACT_SPECS global property beside their install rules
(daslib, dasVulkan, dasMetal, dasSpirv, dasLLAMA); the root processor
extracts at build time and installs the excerpts, and both leak gates
admit only banner-marked ARCHITECTURE*.md. The smoke test now lints
daslib with LINT026 armed and adds a closure lint per module excerpt -
the bundle proof that every shipped citation resolves and every shipped
anchor is cited. section_text moves from utils/mcp/tools/arch_common.das
to utils/common/arch_citations.das (public re-export, no call-site
changes) so the extractor and the tools share one resolution semantics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014mq9AKyoAwvD5T1cigTnRw
Copilot AI review requested due to automatic review settings August 31, 2026 21:57
@borisbat
borisbat force-pushed the bbatkin/sdk-arch-extract branch from 07bf3a8 to 8b881de Compare August 31, 2026 21:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Comment thread CMakeLists.txt
@borisbat
borisbat merged commit 86b0b67 into master Aug 31, 2026
40 checks passed
@borisbat
borisbat deleted the bbatkin/sdk-arch-extract branch August 31, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants