Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 46 additions & 7 deletions docs/languages.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ on interface nodes stores the expected method set for implementation matching.
| Dart | Full | Full | Classes/Enums/Mixins/Extensions | Abstract interface | Full | Full | Full |
| OCaml | Full | Full (class) | Types/Modules | Module types | open | Full | Full |
| Lua | Full | Full (M.func/M:method) | - | - | require() | Full | Full |
| Julia | Full (long + short form, `where` syntax) | Full (qualified `Base.show`, operators) | Structs/abstract/primitive + fields | - | Full (`using`/`import`/`include`, selective lists) | Full (incl. broadcast, macro calls) | `const` |
| Julia | Full (long + short form, `where` syntax) | Full (qualified `Base.show`, operators) | Structs/abstract/primitive + fields | - | Full (`using`/`import`/`include`, selective lists incl. macros/operators) | Full (incl. broadcast, macro calls, `Vector{Int}` constructors) | `const` (with `member_of`) |

### Rust specifics

Expand Down Expand Up @@ -113,7 +113,11 @@ Recent extraction refinements (each covered by a per-feature CI golden): Java `@

`module` / `baremodule` index as `KindType` nodes (the graph's `KindModule`
is reserved for ecosystem packages) and carry the module's `export` list in
`Meta["exports"]`; definitions inside a module get `member_of` edges to it.
`Meta["exports"]` — including exported macros and operators, recorded
verbatim (`export @m, ⊗` records `@m` and `⊗`) — and the Julia 1.11
`public` list in `Meta["public"]` (public-without-reexport, operators
included); definitions, constants and
nested modules inside a module get `member_of` edges to it.
Node ids stay flat — the enclosing module rides on `Meta["scope_mod"]`, the
Rust `mod` convention — and two definitions that would collide on one id
(`f` in module `A` and `f` in module `B`) separate through the shared
Expand All @@ -139,18 +143,53 @@ the unaliased spelling produces. Nothing in the resolver reads the import
`Meta` — the edge targets are the consumable surface. All imports including
`include("file.jl")` target `unresolved::import::<path>`.
Calls attribute to the enclosing function-like definition (long form, short
form, macro, or nested closure) and cover qualified (`Mod.f`) and broadcast
(`f.(x)`, `Meta["broadcast"]`) callees plus macro invocations
(`Meta["macro"]`). Operator calls (`a + b`) are not emitted — dispatch on
operators is not statically attributable. Docstrings attach as
form, macro, or nested closure) and cover qualified (`Mod.f`),
parametric-constructor (`Vector{Int}(xs)` → `unresolved::Vector{Int}`), and
broadcast (`f.(x)`, `Meta["broadcast"]`) callees, plus bare and
module-qualified macro invocations (`Base.@time` → `unresolved::Base.time`,
`Meta["macro"]`). A callee chained onto a call result (`get(cfg).run(x)`)
has no decodable receiver and records its method name (`unresolved::run`);
quoted operators normalise (`Base.:+` and `Base.:(==)` →
`unresolved::Base.+` / `unresolved::Base.==`). Docstrings attach as
`Meta["doc"]` to long and short definitions, types, modules and constants,
but only when the string sits immediately above the documented object —
the adjacency Julia itself enforces, where a blank line or an own-line
comment detaches the string and leaves the definition undocumented. A
string at the top of a function body is executable code, not
documentation. The stored text is the first PROSE paragraph, skipping the
documentation. The explicit `@doc "text" object` form (which is what
Julia lowers every docstring to) attaches the same way, with the text
taken from inside the macro call. The stored text is the first PROSE
paragraph, skipping the
indented signature block Julia's convention opens a docstring with.

What is **not** covered:

- **Calls inside anonymous functions and do-blocks**
(`double = x -> f(x)`, `map(xs) do y g(y) end`) attribute to the
ENCLOSING function — source locality outranks a closure node for the
graph's consumers, so the closure itself mints no node. A short-form
definition nested in a block (`nested() = 1`) is still its own node.
- **`@enum` members** are not extracted — the macro generates the enum
type and its member constants at runtime.
- **`@.`** records no macro edge (a target named `.` is meaningless);
calls inside its arguments edge normally.
- **Operator calls in infix position** (`a + b`) — dispatch on operators is
not statically attributable. Explicit call syntax (`Base.:+(a, b)`) is a
normal call.
- **Typed constant declarations** (`const X::Int = 1`) mint no variable node.
- **Parametric constructor definitions** (`Box{T}(x) where T = …`) are
extracted as plain functions named `Box{T}` — not with the
`<Type>.<init>` constructor spelling, and not bound to `Box`.
- **Callable-object definitions** (`(f::Box)(x) = …`) are not extracted.
- **Calls inside string interpolation** (`"$(f(x))"`) are not walked.
- **Two methods of one name on one physical line**
(`g(x) = h(x); g(y) = k(y)`) collapse onto one node — line numbers cannot
separate them — though each body's call edges are preserved.
- **Call and macro edges are extraction-side facts**: targets are
`unresolved::` names, and binding them to definitions in another file —
including qualified calls into another file's module, and constructor
call sites to `<Type>.<init>` — is resolver work, not attempted here.

## Data, config, build

| Language | Extensions | What it extracts |
Expand Down
2 changes: 1 addition & 1 deletion internal/indexer/extractor_version.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ var extractorVersions = map[string]int{
"go": 3, // generic instantiations are marked so indexing a func value cannot bind (was: generic calls emit call edges)
"cpp": 2, // templated and namespace-qualified calls emit call edges
"swift": 2, // generic calls and ordinary member calls emit call edges
"julia": 2, // bespoke tree-sitter extractor replaces the regex extractor (fields, exports, qualified methods, broadcast/macro calls)
"julia": 3, // callee and macro decoding from CST children (chained/parametric callees, Base.@time, Base.:(==)), macro/operator exports, member_of for module consts and nested modules, @doc and public metadata (was: bespoke tree-sitter extractor replaced the regex extractor)
}

// extractorSaltExtLang maps a lower-case file extension to the language
Expand Down
10 changes: 9 additions & 1 deletion internal/indexer/extractor_version_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,14 @@ func TestStaleLangsDetection(t *testing.T) {
snapshotJSON(t, nil, "julia")); !reflect.DeepEqual(got, []string{"julia"}) {
t.Errorf("previous-release snapshot without julia = %v, want [julia]", got)
}
// A store extracted by the previous Julia extractor version must
// re-extract unchanged .jl files too: the callee decoder, macro
// and export handling, containment edges, and docstring metadata
// all changed what the graph records without any content change.
if got := ExtractorVersionStaleLangs(
snapshotJSON(t, map[string]int{"julia": 2})); !reflect.DeepEqual(got, []string{"julia"}) {
t.Errorf("snapshot stored at the previous julia version = %v, want [julia]", got)
}
if got := extractorVersionsSnapshot()[postExtractionPolicySnapshotKey]; got != postExtractionPolicyVersion {
t.Errorf("persisted policy epoch = %d, want %d", got, postExtractionPolicyVersion)
}
Expand Down Expand Up @@ -238,7 +246,7 @@ func TestStaleLangsDetection(t *testing.T) {
// The Merkle half of the Julia bump: without the .jl → julia
// mapping the leaf salt stays empty and Merkle mode misses the
// bump the same way the mtime path did.
if got, want := merkleSaltFor("src/model.jl"), policySalt+"|julia@2"; got != want {
if got, want := merkleSaltFor("src/model.jl"), policySalt+"|julia@3"; got != want {
t.Errorf("Julia extractor salt = %q, want %q", got, want)
}
})
Expand Down
Loading
Loading