Make the ShEx vocabulary and ShExR's renderings derived, and keep them in sync

[ericprud]

Six commits. The theme is that several files here were copies of each other maintained by hand, and they had drifted; this makes each one derived from a single source and adds checks to npm test so they cannot drift again.

vocab/ — the source of http://www.w3.org/ns/shex

shex.ttl, shex.jsonld and shex.html at w3.org/ns are generated from a vocab.csv by a script Gregg Kellogg (RIP) wrote for the purpose. That source had been sitting in the long-dormant shexspec.github.io repo. It now lives in vocab/, ported from Ruby to dependency-free Node and verified to reproduce the Ruby's output byte-for-byte before any change was made.

It lands in this repo because this repo already carried two hand-maintained copies of that vocabulary, and both had drifted:

• doc/ShExJ-context.jsonld was the @context of shex.jsonld, copied. All 70 keys matched exactly; it was missing only shex:extends. It is now generated (npm run vocab).
• doc/ShExR.shex ran ahead of the published vocabulary, using sx:ShapeDecl, sx:abstract, sx:imports — none of which had ever been published — and sx:negated.

npm run vocab-check reports terms ShExR uses that the vocabulary does not define, and is part of npm test.

The vocabulary side of this is w3c/ns#38, which adds those three terms plus the shape-map terms.

sx:negated removed

negated was the ShEx 2.0 draft's ! operator on TripleConstraint. It was removed from the language in November 2016 — 224c610 here, d3f882d in spec — over the semantics problem in shexSpec/shex#11, and replaced by {0,0} cardinality and ShapeNot. Seven weeks later fe51e15 ("~ aligned with ShExJ.jsg") copied it back into doc/ShExR.shex from the grammar, which had not been cleaned up yet. e01f012 ("TripleConstraint -= negated:BOOL") dropped it from doc/ShExJ.jsg in 2020 but left ShExR alone, so the RDF rendering kept a term the JSON rendering had deleted.

It was never in the published namespace, is not in the spec, and the ShExC parser has no ! rule — nothing could produce it. Gone from ShExR.{shex,ttl,json,ntriples}, plus doc/ShExV.jsg and the schemas/meta.ttl sameAbstractSyntaxAs line referencing two fixtures deleted in 2016. shexSpec/spec#(companion PR) and shex.js 489a1a75 cover the copies elsewhere.

bin/mkShExR.js — ShExR's RDF renderings are now derived

doc/ShExR.shex is the hand-maintained source; the other three files in doc/ are the same schema in other syntaxes, hand-synced on the honour system. doc/ShExR.ntriples was last regenerated in 2017 (7a7da0d) — 800 triples against ShExR.shex's 1341, predating ShapeDecl, abstract, imports and extends entirely.

bin/mkShExR.js parses ShExR.shex to ShExJ and expands it through doc/ShExJ-context.jsonld, since ShExJ + that context is ShExR. So this also exercises the context, which is generated from vocab/vocab.csv — the vocabulary and the schema-of-schemas are now chained together.

• ShExR.ttl and .ntriples are regenerated and byte-compared by CI (npm run shexr-check).
• ShExR.json is compared as an AST rather than regenerated, so its hand formatting survives. It was already exactly what ShExR.shex parses to.
• No content changed in ShExR.ttl — canonicalising the previously committed file yields the regenerated one exactly. That diff is formatting; the .ntriples diff is nine years of catch-up.

The Turtle is pretty-printed rather than handed to N3.Writer, which labels every blank node and spells out all 265 rdf:first/rdf:rest triples. The ShExR graph is a tree, so blank nodes nest as [ ] and lists collapse to ( ). Subject order comes from the rendered name, never a blank node label, so output does not depend on which parser produced the quads; the printer is idempotent.

doc/syntax-deltas.html

Records where ShExC, ShExJ and ShExR disagree, found while doing the above, with links into the ShExC productions and both grammars here. The substantive one: ShExR's sx:start @<#shapeDeclOrExpr> admits a ShapeDecl at start, which ShExJ.jsg rejects and no ShExC production denotes — verified against all three. ShExR has no choice, since RDF cannot distinguish reference from containment, and it cannot even be narrowed to IRIs because shapeDeclLabel admits BNODE.

Checks

npm test passes. test-ts remains broken independently of this PR — it shells out to bin/makeTsTests.sh and calls a bare tsc with no typescript devDependency; it is excluded from CI for that reason and unchanged here.

🤖 Generated with Claude Code