Context
#116 adds Ruby extraction via @ruby/prism rather than a vendored tree-sitter-ruby grammar. The PR is well-built and the accuracy argument for Prism is sound — it's Ruby core's own parser, maintained by the Ruby core team, and more semantically accurate for Ruby than the community tree-sitter grammar. Ruby was previously silently skipped entirely, so this closes a real coverage gap.
This issue is not about that PR's quality. It's about a decision the PR forces, which should be made explicitly rather than as a side effect of merging: do we support more than one parser architecture?
What #116 introduces
Every other language in the repo goes through one path: a vendored .wasm tree-sitter grammar in grammars.ts's WASM_GRAMMAR_FILES, parsed by parse() into a TSTree, handed to a LanguageExtractor. #116 adds a parallel path for one language:
src/graph/extraction/prism-runtime.ts — a second parser loader alongside the tree-sitter one, with its own WASM shipped inside the npm package rather than vendored@ruby/prism ^1.9.0 as a new runtime dependencyrubyExtractor.extract() ignores its tree: TSTree parameter and re-parses source with Prism directlygrammars.ts#parse() returns a placeholder TSTree for Ruby so extractFile's generic if (!tree) return null gate looks uniform across languages
The frozen LanguageExtractor interface, node identity scheme, and SQLite schema are all untouched — deliberately, and the deviations are documented in docs/extractors.md. That's the right instinct and it's why this is a design question rather than a review blocker.
The concern
The placeholder TSTree makes the uniformity cosmetic. extractFile's null gate still fires identically on the happy path, but a Prism parse failure now surfaces inside extract() instead of at the gate — so Ruby's error path genuinely diverges from every other language even though its success path doesn't. Anything downstream reasoning about "parse failed" as a single condition now has a Ruby-shaped hole.
Second, this is a precedent. The next contributor who wants a non-tree-sitter parser (a Python one via CPython's own AST, a Go one via go/parser) will cite this PR, and the argument will be equally good each time. Deciding it once here is cheaper than deciding it five times ad hoc.
Options
A. Merge #116 as-is. Accept Prism, accept the precedent, keep the deviation documented as an exception. Fastest path, best Ruby accuracy today. Cost: two parser architectures, a cosmetic null gate, and a precedent we'd be re-litigating later.
B. Ask for tree-sitter-ruby instead. Full architectural consistency, no new dependency, no placeholder tree. Cost: worse Ruby semantics, and it discards work that's already done and verified against a 1600-file Rails codebase. Hard to justify on merit alone.
C. Generalize the abstraction first. Introduce a real parser-backend seam so non-tree-sitter parsers are first-class — extract() takes a parse result the backend defines rather than a TSTree it may ignore, and the null gate moves to the backend. Then land Ruby on it. Correct long-term, and it removes the placeholder entirely. Cost: touches the frozen LanguageExtractor interface, so it's a core change under CONTRIBUTING.md:19, and it blocks #116 behind refactor work that isn't the contributor's to do.
My leaning
C is right if we expect more non-tree-sitter parsers; A is right if Ruby is genuinely special. Prism is unusual in being a first-party parser with an official WASM build — most languages don't have that, which is an argument that Ruby really is the exception rather than the first of many.
Tentatively: A now, C when a second non-tree-sitter parser shows up, with the deviation note in docs/extractors.md treated as a standing marker rather than a footnote. But I want to hear disagreement before acting on it, particularly on whether the divergent error path causes problems in extractFile that I'm not seeing.
/cc @emmahyde @abhinav-phi
Context
#116 adds Ruby extraction via
@ruby/prismrather than a vendoredtree-sitter-rubygrammar. The PR is well-built and the accuracy argument for Prism is sound — it's Ruby core's own parser, maintained by the Ruby core team, and more semantically accurate for Ruby than the community tree-sitter grammar. Ruby was previously silently skipped entirely, so this closes a real coverage gap.This issue is not about that PR's quality. It's about a decision the PR forces, which should be made explicitly rather than as a side effect of merging: do we support more than one parser architecture?
What #116 introduces
Every other language in the repo goes through one path: a vendored
.wasmtree-sitter grammar ingrammars.ts'sWASM_GRAMMAR_FILES, parsed byparse()into aTSTree, handed to aLanguageExtractor. #116 adds a parallel path for one language:src/graph/extraction/prism-runtime.ts— a second parser loader alongside the tree-sitter one, with its own WASM shipped inside the npm package rather than vendored@ruby/prism^1.9.0as a new runtime dependencyrubyExtractor.extract()ignores itstree: TSTreeparameter and re-parsessourcewith Prism directlygrammars.ts#parse()returns a placeholderTSTreefor Ruby soextractFile's genericif (!tree) return nullgate looks uniform across languagesThe frozen
LanguageExtractorinterface, node identity scheme, and SQLite schema are all untouched — deliberately, and the deviations are documented indocs/extractors.md. That's the right instinct and it's why this is a design question rather than a review blocker.The concern
The placeholder
TSTreemakes the uniformity cosmetic.extractFile's null gate still fires identically on the happy path, but a Prism parse failure now surfaces insideextract()instead of at the gate — so Ruby's error path genuinely diverges from every other language even though its success path doesn't. Anything downstream reasoning about "parse failed" as a single condition now has a Ruby-shaped hole.Second, this is a precedent. The next contributor who wants a non-tree-sitter parser (a Python one via CPython's own AST, a Go one via
go/parser) will cite this PR, and the argument will be equally good each time. Deciding it once here is cheaper than deciding it five times ad hoc.Options
A. Merge #116 as-is. Accept Prism, accept the precedent, keep the deviation documented as an exception. Fastest path, best Ruby accuracy today. Cost: two parser architectures, a cosmetic null gate, and a precedent we'd be re-litigating later.
B. Ask for
tree-sitter-rubyinstead. Full architectural consistency, no new dependency, no placeholder tree. Cost: worse Ruby semantics, and it discards work that's already done and verified against a 1600-file Rails codebase. Hard to justify on merit alone.C. Generalize the abstraction first. Introduce a real parser-backend seam so non-tree-sitter parsers are first-class —
extract()takes a parse result the backend defines rather than aTSTreeit may ignore, and the null gate moves to the backend. Then land Ruby on it. Correct long-term, and it removes the placeholder entirely. Cost: touches the frozenLanguageExtractorinterface, so it's a core change underCONTRIBUTING.md:19, and it blocks #116 behind refactor work that isn't the contributor's to do.My leaning
C is right if we expect more non-tree-sitter parsers; A is right if Ruby is genuinely special. Prism is unusual in being a first-party parser with an official WASM build — most languages don't have that, which is an argument that Ruby really is the exception rather than the first of many.
Tentatively: A now, C when a second non-tree-sitter parser shows up, with the deviation note in
docs/extractors.mdtreated as a standing marker rather than a footnote. But I want to hear disagreement before acting on it, particularly on whether the divergent error path causes problems inextractFilethat I'm not seeing./cc @emmahyde @abhinav-phi