Feat/julia extractor followups - #694
Merged
Merged
Conversation
added 12 commits
August 29, 2026 01:09
get(cfg).run(x) reached the graph as the call target
`unresolved::get(cfg).run` — argument text and all — because
juliaCalleeName decoded a qualified callee by splitting its source text
on the last dot. A chain broken across lines carried the line break
itself into the target:
run(x) = foo(x,
1,
).bar(y) # target "unresolved::foo(x,\n 1,\n).bar"
and Base.:(==)(a, b) normalised to `Base.(==)` while Base.:+(a, b)
normalised to `Base.+`, because the parenthesised quote form survived
the text trim.
Decode the field_expression's base and property children instead. A
property keeps its operator's own spelling (`+`, `==`) whether Julia
wrote it as `:+` or `:(==)`, and a bare `(==)(a, b)` callee — which
wears only the parentheses — decodes the same way. When the base of a
chain is not itself a dotted name (a call result, as above) only the
method name is decodable, which is the only part a resolver could ever
match, so the callee degrades to its bare name rather than leaking
argument text into the graph; a genuinely dotted base (A.B.c) keeps its
full qualification.
Vector{Int}(xs) produced no call edge: the callee is a
parametrized_type_expression and the callee decoder had no case for
it, so every construction of a parametric type — one of the most
common calls in real Julia — vanished from the call graph, while the
equivalent unparametrized Vector(xs) was recorded fine.
Decode the head (Vector, or Base.Vector qualified) plus the literal
type parameters, so the edge names the constructor the way Julia
itself prints it: build -> unresolved::Vector{Int}, qualified ->
unresolved::Base.Vector{Int}. The parameter list is rebuilt from its
child nodes rather than source text, so a { ... } broken across lines
cannot leak a newline into the target.
Base.@time helper(xs) recorded helper's call edge but never the macro's own: handleMacroCall matched only a direct macro_identifier child of the macrocall_expression, and a qualified macro nests that identifier under a field_expression (Base.@time), which the scan never opened. Every Distributed- or Base-qualified macro in a function body was thus invisible to the call graph while its unqualified twin was recorded. Open the field_expression too, and emit the macro edge for the qualified form with the module as receiver — target Base.time with macro:true meta, the same receiver.name spelling qualified call callees already use. The module-alias rewrite applies as well, so `import Foo as F` followed by F.@Spawn attributes to Foo.spawn, matching what a qualified call in the same position does.
A docstring above a macro call switches the macro-argument walker into
its doc-carrying loop, which dispatches definition arguments to their
handlers but walked everything else with walk() — and walk() visits a
node's children, never the node itself, so a call_expression argument
was never shown to the call handler:
module M
"""load helpers on every worker"""
@Everywhere include("helpers.jl") # import edge lost
end
The undocumented form of the same statement worked, because the
generic walker dispatches each child's own kind. include() at module
level is the case that observably regressed: ordinary call edges need
an enclosing function, which a documented — module- or file-level —
macro call never has.
Dispatch a macro argument's own kind before walking into it, exactly
as the generic walker does for calls, broadcast calls and nested
macro calls.
export apply, @m, ⊗ recorded only `apply`: the export scan took identifier children of the export statement, but a macro name is a macro_identifier node and an operator an operator node — the same distinction import lists already decode (using Base: @time, + is handled). A module whose public surface is macros or operators reported an exports list that silently omitted them. Decode all three node kinds, recording the macro verbatim (@m, not m) so the recorded name matches what `names(MyModule)` would answer in Julia itself.
const X = 1 inside module M, and module Inner inside module Outer,
reached the graph with their lexical module recorded on scope_mod but
with no member_of edge — the only residents of a module a traversal
from it could not reach, since every function, type and field already
carries one:
module Outer
const X = 1 # invisible from Outer
module Inner ... end # invisible from Outer
end
scope_mod on Meta is a fact about lexical scope, not a traversable
edge; emit the same EdgeMemberOf the callable and field paths emit, so
constants and submodules join the module's graph neighbourhood. At the
top level, where there is no enclosing module, neither gets one.
The Julia row advertised plain "Full" cells for imports, calls and
constants while the extractor did not record exported macros/operators,
parametric constructor calls, module-qualified macro invocations, or
containment edges for constants and nested modules — and several
deliberately dropped forms (typed consts, callable-object definitions,
calls in string interpolation, same-line method twins, the
extraction-side-only status of every call target) had no exclusion
sentence at all.
Enumerate the now-covered forms in the matrix cell and the
Julia-specifics prose, and state the exclusions explicitly, Rust-row
style: each dropped form gets a sentence saying exactly what happens
instead (e.g. a parametric constructor definition mints a plain
function named Box{T}, not a <Type>.<init> node; a chained callee
records its method name; call edges stay unresolved:: targets and
cross-file binding remains resolver work).
function Base.show(io, x) in a file that declares no Base emitted a member_of edge to <file>::Base — a node-shaped id for which no node exists. The edge claimed a resident of the graph that was not there, while every other edge that can point past the file (extends to a supertype, calls, imports) already marks such targets with the unresolved:: prefix. Keep the method's own id flat — ids are spellings, and the flat form is what the owner derivation and the Lua M.func convention use — but point the member_of at unresolved::Base when no type in the file (or its lexical module scope) provides the receiver. An in-file receiver still targets the real type node, and constructors still target the type they build.
@doc "Short doc." pd(x) = x extracted pd but not its documentation: the doc-carrying macro-argument walk only looked for a docstring ABOVE the macro call, while the explicit form — which is what Julia lowers every docstring to, per the manual's own lowering of "str" obj to Core.@doc — carries the string INSIDE the call, beside the object it documents. The most documented-thing-shaped macros in real code (@doc over struct/function/module/short-form) came out undocumented whenever the author used the explicit spelling. Recognise the doc form (bare @doc or qualified Core.@doc / Base.@doc): the first string argument becomes the doc, normalized by the same first-prose-paragraph rule as the implicit form, and the object beside it is dispatched with it — including the short-form assignment shape, which the definition-only dispatch missed. An @doc with a string and no object attaches nothing.
Julia 1.11 added `public` for naming API that is visible without being re-exported — a different contract from `export`, and until now the statement parsed but recorded nothing, so a module's users-visible surface was silently incomplete on modern Julia. `public +` is legal (operator names are ordinary here), and the statement's children are the same node kinds the export scan already decodes. Share the export recorder under a Meta key: exports stay on Meta["exports"], public names land on Meta["public"], both verbatim (operators and macros included, @m spelled with the @).
…aries Three deliberately-uncovered shapes had no exclusion sentence, so the docs neither claimed nor declined them: calls inside anonymous functions and do-blocks attribute to the ENCLOSING function (source locality for the graph's LLM consumers outranks a closure node — a design decision worth stating, since it is the opposite trade-off from the nested short-form closure, which is its own node); @enum members generate no nodes; and the @. broadcast macro records no macro edge while its arguments' calls edge normally. Each now says what happens instead, Rust-row style.
A graph extracted by the previous Julia extractor silently misses facts
that no content change will ever re-trigger:
get(cfg).run(x) # call target carried the raw source text
Base.@time helper(xs) # no macro edge at all
export @m, ⊗ # macro and operator names missing from exports
const X = 1 # no member_of edge to the enclosing module
@doc "Doc." f(x) = x # Meta["doc"] never attached
public pf1, pf2 # Meta["public"] never recorded
Raise the version so every stored repository re-extracts its .jl files
on the next full-root incremental pass; the version rides the Merkle
leaf salt, so unchanged content is not even re-read and no other
language is disturbed.
Known limitation, unchanged here: the non-Merkle staleness gate runs
only on a full-root incremental pass — a warm-started daemon never
consults it. Pre-existing for every bumped language.
zzet
requested changes
Aug 29, 2026
zzet
left a comment
zzet
left a comment
Owner
There was a problem hiding this comment.
Requesting changes for six confirmed issues:
-
P1 — CI is blocked by an ineffectual assignment.
internal/parser/languages/julia.go:895assignsownerTarget = ownerID, but every path immediately overwrites it. This reproduces the currentineffassignfailure locally and in CI. -
P1 — Same-file module methods get an unresolved owner. At
julia.go:896, qualified receivers are resolved only throughlookupType. Modules live inst.nodes, notst.types, somodule M ... end; function M.f() ... endemitsmember_of -> unresolved::Malthoughfile::Mexists. Track lexical modules and add an assertion forM.f -> file::M. -
P2 — Explicit documentation drops constants.
julia.go:320does not dispatch a directconst_statement, so@doc "text" const X = 1reaches assignment handling withoutisConst, dropping the constant and its documentation. -
P2 — Multi-segment qualified macros lose their call edge.
julia.go:1361requires the receiver to be a single identifier. Valid calls such asBase.Threads.@threadsandA.B.@mtherefore disappear. Decode dotted receivers recursively and cover a three-segment qualifier. -
P2 — Arbitrary qualified
@docmacros are misclassified.julia.go:280checks only the property name.Foo.@docconsequently attachesMeta["doc"]; restrict recognition to the supported bare/Core/Base documentation macros and add a negative test. -
P2 — Nested parametric callees retain raw source formatting.
julia.go:722canonicalizes only the outer parameter list. A valid nested type such asVector{Tuple{Int,\nString}}(x)can still create a newline-dependent unresolved target. Canonicalize nested type expressions recursively.
The focused parser and indexer tests pass, but lint fails and none of the semantic cases above currently has regression coverage.
added 6 commits
August 29, 2026 12:38
zzet
approved these changes
Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #675, addressing point 8 ("Cover the advertised 'Full' forms or narrow
the documentation") and the decoding bug zzet left for the
Julia AST work. Both done: the extractor now handles the
previously-missing call, macro, and export/containment shapes, and
docs/languages.mdis narrowed to enumerate exactly what is and isn't covered. Bumps the Julia extractor
version to 3 so existing graphs re-extract.
Changes
(
juliaCalleeName/ newjuliaUnwrappedName):get(cfg).run(x)now recordsunresolved::runinstead ofunresolved::get(cfg).run, and a chain broken acrosslines no longer leaks a newline into the target. Quoted operators normalise —
Base.:+→unresolved::Base.+,Base.:(==)→unresolved::Base.==.Vector{Int}(xs)→unresolved::Vector{Int})via new
juliaParametrizedCallee, rebuilding the{…}list from children so amulti-line parameter list can't leak a newline into the target.
Base.@time f(x)→unresolved::Base.time,Meta["macro"]), including alias resolution (import Foo as F; F.@spawnnamesFoo).export @m, ⊗records@mand⊗),plus the Julia 1.11
publiclist inMeta["public"](
handleExport/handlePublicrefactored onto a sharedrecordModuleNames).member_offor module-level constants and nested modules(previously only functions/types carried them; constants carried
scope_modon Metaonly, which is not traversable).
member_ofhonesty:function Base.showinside a module pointsits
member_ofatunresolved::Baserather than a node-shaped id for a type this filedoesn't declare.
@doc "text" objectdocstrings attach the same way triple-quoted ones do,and call sites inside documented macro arguments keep their edges.
docs/languages.md: Julia table row enumerates the newly-covered forms; the Juliasection gains a "What is not covered" list (parametric constructor definitions,
callable objects,
const X::Int, string-interpolation calls, same-line twins) followingthe Rust-row house style.
Testing
go test -race ./...) — branch related packages green(
internal/parser/languages,internal/indexer);(
internal/gitcmd,internal/agents/opencode) fail only under full-suite parallel load and pass on isolated rerun, same as noted in Replace regex extractor with tree-sitter for Julia #675. Neither touches Julia.Checklist
scope_mod,<Type>.<init>constructor spelling, JS/TS per-binding import cap, python
base_pathconvention)Meta["methods"]for interfaces (N/A — Julia has none)Cheers