fix: resolve @property provenance through CTE bodies, correct-or-silent - #243
Merged
Marius Volkhart (MariusVolkhart) merged 1 commit intoAug 22, 2026
Merged
Conversation
A nullable result-class property shipped with no @Property KDoc line at all when it was both CTE-wrapped and expression-derived. `parseSelectItems` deliberately windows past the WITH clause, so the outer item is a plain reference; pgjdbc reports an empty table name for such a column. Both inputs to `sourceReference()` were empty, so the line was filtered out entirely -- exactly the case the project's "document what null means" rule exists for. Provenance now resolves through the CTE definition rather than the outer select list, via `resolveCteOutputExpression`. The design is a whitelist, not a blacklist, and that choice is load-bearing. A general resolver was written first -- multi-CTE, alias-addressed, JOIN-aware, DML main queries -- and it emitted *wrong* provenance across a long tail of valid SQL: column-list CTEs invisible to the ambiguity scan, FROM aliases shadowing another CTE's name, DML target relations treated as CTE sources, INSERT RETURNING resolving against the feeding FROM instead of the insert target, set operations reporting only the first branch. A wrong `@property x (`SOME_EXPR`)` is worse than a missing line, so resolution is now restricted to a shape whose correctness argument is one sentence: exactly one CTE declared, and the main query's FROM is exactly that CTE. Under that precondition sibling ambiguity cannot arise. Everything else returns null and reproduces the old behaviour -- a missing line, never a wrong one. PgCatalogLoader states the same principle: string-based SQL transformation cannot be proven exhaustively correct for every input shape. Fixing the reported trailing-comment and quoted-alias gaps required widening `COLUMN_REFERENCE` to match quoted identifiers, which corrects two pre-existing wrong outputs on the shared path. `SELECT "Foo" AS bar` derived `originalName` from pgjdbc's `getColumnName` -- the alias `bar` -- so `catalog.findColumn` looked up a column that does not exist and silently dropped the Postgres column comment; it now resolves through the quoted source. A quoted CTE alias echoed its own name back as its provenance instead of reporting the expression. Identifier folding is ASCII-only, matching `downcase_identifier`. Kotlin's `String.lowercase()` applies full Unicode mapping, which disagrees with Postgres about which column an unquoted non-ASCII reference targets -- `SELECT Ü` matches a column named "ü" under Unicode folding but not in Postgres. Mechanical changes: - `resolveCteOutputExpression` in SqlUtils.kt, called from TypeRepository only where `column.table == null && selectItem.columnName != null` - `parseAliasToken`: reads one identifier token after AS, escape-aware for quoted names, rejecting trailing content rather than discarding it - `COLUMN_REFERENCE` matches quoted identifiers; `SelectItem` carries `isColumnNameQuoted`/`isTableNameQuoted` so folding can distinguish quoted-exact from unquoted-ASCII-folded comparisons - `foldAsciiCase`, plus `PG_RESERVED_KEYWORDS` (pg_get_keywords R+T) so a bare reserved word is never matched as a column - `formatAsKdocPropertyReference`: backtick-wraps @Property names that are not plain identifiers. A space-containing name previously emitted two bare tokens, corrupting the tag - golden coverage for quoted column references, including a quoted column carrying a Postgres comment Scenarios beyond the single-CTE whitelist are tracked in #238. Fixes #229 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Marius Volkhart (MariusVolkhart)
requested review from
Matthew Foster (MatthewFoster624) and
Ryan LuMaye (RyanLuMaye)
as code owners
August 22, 2026 14:48
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This was referenced Aug 22, 2026
Marius Volkhart (MariusVolkhart)
deleted the
fix/229-cte-expression-property-kdoc
branch
August 22, 2026 15:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #229.
A nullable result-class property shipped with no
@propertyKDoc line at all when it was both CTE-wrapped and expression-derived.parseSelectItemsdeliberately windows past theWITHclause, so the outer item is a plain reference, and pgjdbc reports an empty table name for such a column — both inputs tosourceReference()were empty, so the line was filtered out entirely.Provenance now resolves through the CTE definition rather than the outer select list.
The design is a whitelist, and that is the point
A general resolver was written first — multi-CTE, alias-addressed, JOIN-aware, DML main queries. It emitted wrong provenance across a long tail of valid SQL. Five adversarial verification rounds, each probing the real function and checking answers against a live PostgreSQL 18, each closed the reported defects and found new ones:
FROMalias shadowing a different CTE's name resolved to the wrong CTEDELETE FROM xto the real table, never a CTE)INSERT ... SELECT ... FROM cte RETURNINGresolved against the feedingFROMinstead of the insert targetA wrong
@property x (`SOME_EXPR`)is worse than a missing line, so resolution is now restricted to a shape whose correctness argument fits in one sentence: exactly one CTE declared, and the main query'sFROMis exactly that CTE. Under that precondition sibling ambiguity cannot arise at all, and ~400 lines of ambiguity/address-map/JOIN machinery was deleted rather than gated. Everything else returnsnulland reproduces the old behaviour — a missing line, never a wrong one.PgCatalogLoaderalready states the principle this converged on: string-based SQL transformation cannot be proven exhaustively correct for every input shape.Shapes outside the whitelist are tracked in #238, which also records why a bigger hand-written parser is the wrong direction and what was already investigated on the Postgres-authoritative path.
Two pre-existing wrong outputs fixed on the way
Fixing the reported trailing-comment and quoted-alias gaps required widening
COLUMN_REFERENCEto match quoted identifiers. That corrects two existing defects, both proven by building the generator at the merge base and running the same inputs through it:SELECT "Foo" AS bar FROM tq(`tq.bar`), comment silently dropped(`tq.Foo`)with its Postgres column comment(`"descriptionUpper"`)— echoed its own name(`UPPER(description)`)The first one mattered:
originalNamecame from pgjdbc'sgetColumnName, which returns the alias, socatalog.findColumnlooked up a column that does not exist and quietly lost the comment.Generating the space-containing-name test case also surfaced a latent bug —
@property My Colemitted two bare tokens, corrupting the tag.formatAsKdocPropertyReferencenow backticks names that are not plain identifiers. Backticks are the correct escape here, confirmed by readingKDocName.getNameTextRangerather than assuming;[My Col]would not bind.Postgres parity details worth a reviewer's eye
downcase_identifier. Kotlin'sString.lowercase()applies full Unicode mapping and disagrees with Postgres:SELECT Ümatches a column named"ü"under Unicode folding, but Postgres raisescolumn "Ü" does not exist. There is a comment onfoldAsciiCasesaying not to "fix" it back.PG_RESERVED_KEYWORDSispg_get_keywords()categoriesR+T, verified 101/101 against live PG 16/17/18. Deliberately wider than the keywords currently known to be niladic: a reserved word that is not niladic cannot appear as a bare select-list item at all, so including it costs nothing and absorbs future additions (assystem_userwas in PG 16, which is exactly how this bug class reappeared once).""-escape aware and shares one pattern withparseColumnReference, since a divergence between two scanners is what produced an earlier cross-match.Verification
ParseSelectItemscasesparseSelectItemscompared at the merge base vs this branch across all 185 scenario statements: 181 identical, 4 differing, all intended quoted referencesÜ/ü, Turkishİ/ı,ß, Greek final sigma, full-widthA, CyrillicА, and combining vs precomposed formsfoldAsciiCaseforlowercase()flips two answers to Postgres-contradicting values./gradlew buildpasses. (An earlier note here claimed a pre-existinge2e-tests-micronautfailure; that was my local Gradle daemon running JDK 21 against a build that declares JDK 25, not a repo problem. Filed as #246, since the daemon JVM is unenforced and the resulting error is cryptic.)🤖 Generated with Claude Code