Skip to content

fix(AF-936): scope schema introspection to the caller's allow-list - #1088

Merged
babltiga merged 5 commits into
mainfrom
fix/AF-936-scope-schema-introspection
Sep 24, 2026
Merged

babltiga merged 5 commits into
mainfrom
fix/AF-936-scope-schema-introspection

Conversation

@babltiga

Copy link
Copy Markdown
Contributor

Closes #936

What

GET /datasources/{id}/schema — and every user-path caller of DatasourceAdminService.introspectSchema (editor schema tree/autocomplete, table preview, AI analyze-preview, text-to-SQL, MCP get_datasource_schema / validate_sql) — is now scoped to the caller's effective permission for non-admins:

  • tables outside allowed_schemas / allowed_tables are dropped (both empty = unrestricted, as in enforcement);
  • per review with the maintainer, scope extended to proxy: column-level authorization — block, not just mask #935 denied columns: columns on the grant's denied_columns are dropped;
  • foreign keys from/to a denied column, to a hidden table, or to a bare name shared with a hidden table are dropped;
  • no effective permission → 404 DATASOURCE_NOT_FOUND, before any connection is opened.

Admins, introspectSchemaForSystem (async AI analysis, discovery, drift, snapshots, replay) and the JIT request-form endpoint are unchanged. Query enforcement is unchanged.

How

  • core.api.AllowedTables — the allow-list normalize / coveringEntry moved out of workflow.internal.DatasourcePermissionChecker (which now delegates, byte-for-byte), so core can reuse the gate's matcher.
  • core.api.DeniedColumns.deniesColumn — per-column check on the existing proxy: column-level authorization — block, not just mask #935 matching rule.
  • core.internal.SchemaViewPermissionFilter — pure filter. Fails closed where the view cannot know what the gate allows: a bare allowed_tables entry shows a table only when its name is unique across schemas (qualify the entry otherwise). Catalog-qualified entries (proj.ds.t, db.dbo.t) match their trailing schema.table.
  • DatasourceAdminServiceImpl.introspectSchema resolves DatasourceUserPermissionLookupService.findFor and applies the filter.

Docs / website

  • docs/04-api-spec.md, docs/05-backend.md, docs/07-security.md, docs/13-mcp.md
  • website/docs/configuration/datasources/index.html (+ regenerated help-corpus/)
  • No README, config knob, migration, i18n, frontend or e2e change. The covered e2e specs (sample data, ER diagram, denied columns) run as admin, so they are unaffected. No UI change, so no screenshots.

Verification

  • mvn -o verify -Pcoverage: 10,241 tests, 0 failures (first commit). The follow-up commit re-ran SchemaViewPermissionFilterTest, DatasourceAdminServiceImplTest, DatasourceConnectionTestIntegrationTest (real Postgres: restricted analyst sees only customers without email; admin sees everything) and DefaultSampleDataServiceTest, all green.
  • ApplicationModulesTest, ApiPackageDependencyTest, spotless, checkstyle, the help-corpus drift check and the website guard suites all pass (af-verifier).

Review notes

Four reviewers ran (af-verifier, af-reviewer, af-java-reviewer, af-content-reviewer). Fixed in the second commit:

  • af-content-reviewer (Blocker): the website paragraph said names "never reach" users but then listed exceptions, and it named the admin-only Schema tab. Rewritten.
  • af-java-reviewer / af-reviewer: a bare entry exposed same-named tables in other schemas. It now fails closed. FK targets were matched by bare name and could name a hidden table; those FKs are now dropped. Three-part entries were hidden; they now match. The IT now asserts status. The "never disagrees with enforcement" wording is scoped down.

Concerns that remain, for a human to weigh:

  • af-reviewer: engine plugins with non-relational qualification can see fewer objects than their gate allows. Couchbase grants use bucket.scope.collection while its view names schemas by scope; Elasticsearch uses index patterns. This only degrades what they see (enforcement is unchanged). Documented as a limitation.
  • af-java-reviewer, content reviewer: the submission-time AI analysis still builds its prompt from the unfiltered schema, so its free-text comments could name a table outside the grant. The issue scoped this path out; it is documented as a residual.
  • af-java-reviewer (pre-existing): DefaultSampleDataService.targetAllowed accepts a bare allowed_tables entry for a table in any schema. So the table preview can read archive.orders under allowed_tables=[orders] even though the query gate rejects archive.orders. This is not introduced here; flagged for a separate fix.
  • Nits not taken: deniesColumn re-normalizes the deny list on each call; DefaultSampleDataService / DefaultQueryDryRunService still have private normalizeList copies.

DefaultSampleDataService now matches its target through core.api.AllowedTables, so a bare allowed_tables entry no longer admits a schema-qualified preview the query gate rejects; a bare entry covers a qualified target only when the name is unique in the view (#936 rule). DefaultQueryDryRunService drops its duplicate normalizer too.

Refs #1089
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Frontend Test Results

    1 files  ±0    308 suites  ±0   11m 50s ⏱️ +8s
2 628 tests ±0  2 628 ✅ ±0  0 💤 ±0  0 ❌ ±0 
2 629 runs  ±0  2 629 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit ca6f0de. ± Comparison against base commit bb996c3.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Report for Frontend Coverage (frontend)

Status Category Percentage Covered / Total
🟢 Lines 96.03% (🎯 90%) 3658 / 3809
🟢 Statements 94.65% (🎯 90%) 4059 / 4288
🟢 Functions 94.02% (🎯 90%) 1118 / 1189
🟢 Branches 87.69% (🎯 80%) 2430 / 2771
File CoverageNo changed files found.
Generated in workflow #1402 for commit ca6f0de by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Backend Test Results

10 255 tests  +34   10 255 ✅ +34   13m 4s ⏱️ + 4m 15s
 1 140 suites + 2        0 💤 ± 0 
 1 140 files   + 2        0 ❌ ± 0 

Results for commit ca6f0de. ± Comparison against base commit bb996c3.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Backend Code Coverage

Overall Project 94.74% 🍏
Files changed 99.73% 🍏

File Coverage
DatasourcePermissionChecker.java 100% 🍏
SchemaViewPermissionFilter.java 100% 🍏
AllowedTables.java 100% 🍏
DefaultSampleDataService.java 99.47% -0.53% 🍏
DeniedColumns.java 99.42% 🍏
DefaultQueryDryRunService.java 93.73% 🍏
DatasourceAdminServiceImpl.java 90.6% 🍏

…ow-list

fix(AF-1089): match table preview allow-list to the query gate
@babltiga
babltiga merged commit e310bfd into main Sep 24, 2026
35 checks passed
@babltiga
babltiga deleted the fix/AF-936-scope-schema-introspection branch September 24, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

core: scope schema introspection to the caller's allow-list

1 participant