Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
93374f3
feat(runtime): require request-owned query and mutation intent
Oct 1, 2026
c704934
feat(runtime): preserve typed graph lineage through SQL and committed…
Oct 2, 2026
5dc021b
feat(trace): retain generated graph lineage and request-owned relatio…
Oct 2, 2026
1219b2a
fix(trace): retain per-item lineage in Java prepared graph inserts (#…
Oct 2, 2026
aa357b2
fix(trace): retain per-item lineage through prepared updates deletes …
Oct 2, 2026
54a855e
fix(trace): isolate Checker invocations and read-only relation ledger…
Oct 2, 2026
5ca457c
fix(trace): validate atomic mutation routes per plan instead of Conte…
Oct 2, 2026
2d08920
fix(trace): keep query and stream provenance off shared Context (#202)
Oct 2, 2026
a204990
fix(trace): preserve root provenance in derived SQL queries (#202)
Oct 2, 2026
a2b72ca
fix(trace): retain Java mutation statements with logging disabled (#202)
Oct 3, 2026
95a4922
fix: retain graph-local cross-type mutation privacy (#202)
Oct 3, 2026
f2da061
fix: retain materialized query evidence independently of logging (#202)
Oct 3, 2026
181d33d
fix: retain query cursor terminal evidence with logging disabled (#202)
Oct 3, 2026
2fbe5e1
fix: preserve nested loads when projecting relation keys (#202)
Oct 3, 2026
3de0c4a
test: verify request intent diagnostics before execution (#202)
Oct 3, 2026
5c339f4
fix: remove unaudited Portable bootstrap entry points (#202)
Oct 3, 2026
aa1ceec
test: exercise request-owned traces at Java intent gates (#202)
Oct 3, 2026
576a91b
fix: preserve typed LIKE operands in private query intent (#202)
Oct 4, 2026
28a1005
test: verify filtered forward identity through generated Q/E (#202)
Oct 4, 2026
120b48a
test: prove same-context live SQLite query trace isolation (#202)
Oct 4, 2026
9c9889d
test: assert canonical generated three-relation SQL trace paths
Oct 4, 2026
7e7ae1a
test: require exact six-entity command and audit identities (#202)
Oct 4, 2026
cb0ab8e
fix: retain request-owned root intent across Java mutation SQL
Oct 4, 2026
8c5b45e
test: assert complete ledger override leaves generated sibling fallba…
Oct 5, 2026
5560aaa
test: retain complete private root lineage in generated SQL and audit
Oct 5, 2026
7f0deb2
test: retain generated bootstrap request ownership before clearing ev…
Oct 5, 2026
0d514a3
test: assert generated Java readback inherits captured mutation intent
Oct 5, 2026
b5b50b9
fix: preserve nested facet metadata and request-owned membership queries
Oct 5, 2026
319c657
fix: retain independently scoped loaded relation Facet results
Oct 5, 2026
1cedbe7
test: verify generated reverse collection Facet trace and full member…
Oct 5, 2026
ff77320
docs: name the three generated Trace Chain test classes
Oct 5, 2026
38dab2d
test: resolve runtime module for HANA privacy tests
Oct 5, 2026
698b8c1
fix: classify private aggregate predicates before root SQL diagnostics
Oct 5, 2026
67d0f94
fix: classify BETWEEN and phonetic operands before root SQL logs
Oct 5, 2026
9b0156f
test: preserve owned mutation comment through blank route tails at SQ…
Oct 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions DIALECT_INTEGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ TeaQL 的架构分为两层:
- **`teaql-data-service-sql` (JDBC 适配层)**:基于 `SqlExecutionAdapter` 实现的与数据库真实通信的通道。

## 2. 如何实现 `ensureSchema` (表结构同步)
你**不需要**去手工遍历所有列、对比类型、拼接 CREATE/ALTER 语句,这些在 `PortableSQLRepository.ensureSchema()` 中已经完美实现。
你**不需要**去手工遍历所有列、对比类型、拼接 CREATE/ALTER 语句,这些由 `PortableSQLRepository.ensurePhysicalSchema()` 统一实现。

在具体的数据库方言执行器(例如 `PostgresDataServiceExecutor`)中,只需执行以下 3 步:

Expand Down Expand Up @@ -50,7 +50,7 @@ TeaQLDatabase dbAdapter = new TeaQLDatabase() {
```

### Step 3: 交给 Portable 引擎执行 DDL
针对每个 `EntityDescriptor`,实例化一个带有该伪装 Adapter 的 PortableRepository,并调用其 `ensureSchema` 方法:
针对每个 `EntityDescriptor`,实例化一个带有该伪装 Adapter 的 PortableRepository,并调用其 `ensurePhysicalSchema` 方法:
```java
for (EntityDescriptor descriptor : descriptors) {
// 实例化方言的 PortableSQLRepository(例如 PostgresPortableSQLRepository,如果没有则用基类)
Expand All @@ -63,6 +63,12 @@ for (EntityDescriptor descriptor : descriptors) {
不能顺带修改生产数据库。新方言的最小验证应包含两个独立 context/metadata 实例,
证明它们只处理自己的实体,再对目标数据库执行 live schema、查询和审计写入测试。

`ensurePhysicalSchema` 只处理数据库结构,不解释根对象或常量的候选值。
数据播种由 context 调用已安装的 `GeneratedSchemaBootstrap`,使用带有
comment/purpose 的 Q API 和 audited save,经过 Checker、Mutation Policy、
乐观锁及提交后的审计链。旧的 Portable `ensureSchema` / `ensureInitData`
直写数据入口已移除;不要在方言中重建这条绕过路径。

## 3. 核心纪律
1. **彻底解耦 Spring**:在方言模块中,严禁直接使用 `JdbcTemplate` 或任何 `org.springframework` 包。全部通过 `SqlExecutionAdapter` 委托。
2. **职责极简**:方言层(后端层)只负责提供“查询数据字典的原生 SQL”和“JDBC 链接”,表结构的 Diff 对比和通用 DDL 必须收口在 Portable 引擎。
117 changes: 109 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,29 +110,130 @@ Mutations declare an audit action:
task.auditAs("Move task to Done").save(userContext);
```

### Request owned intent on the development branch

The `feature/request-trace-chain` branch makes non-blank `comment` part of both
`QueryRequest` and `MutationRequest`. Query also requires non-blank `purpose`.
Existing generated `.comment(...).purpose(...)` and `.auditAs(...)` spelling
does not change. Low-level provider requests now expose an immutable validated
`QueryIntent` or `MutationIntent`; custom SPI implementations must adopt this
contract. These changes are not a claim about published Maven artifacts.

Missing or Unicode-whitespace-only comment fails with
`REQUEST_COMMENT_REQUIRED` at `comment`; missing Query purpose fails with
`QUERY_PURPOSE_REQUIRED` at `purpose`. Validation happens before policy and
provider execution, including direct runtime calls and disabled logging.
Neither a Context default nor a fabricated trace supplies missing intent.

List, aggregate, relation and streaming execution no longer push or pop query
frames on Context. Streaming providers now accept the same validated
`QueryRequest` envelope as materialized providers instead of a bare
`SearchRequest`. A custom `StreamingQueryExecutor` must migrate that SPI
signature; generated `.executeForStream(context)` calls remain unchanged.
The captured intent survives Policy changes to a builder and delayed cursor
consumption. SQL providers snapshot source paths and redaction provenance for
the invocation, including inherited internal streams. Legacy unbound direct SQL
diagnostics can still use explicitly supplied Context frames; those compatibility
calls are not the runtime query ownership contract.

Derived relation, Facet and materialized relation-predicate queries carry their
validated originating intent instead of asking callers to repeat it. Mutation
reason is captured before policy and retained in provider requests and committed
audit facts. Graph saves now use immutable parent-linked mutation scopes, rather
than a Context push/pop stack. Each persistence request carries its own typed
lineage through SQL writes, authoritative readback and committed safe audit.
`TraceNode` includes entity type and assigned ID; Entity and Ledger trace setters
now accept immutable `List<TraceNode>`, not flattened strings. Custom callers
using the old string API must migrate; this is a local source change, not a
released API. SQL `tracePath` remains separate from `mutationLineage`.

The local runtime tests cover branch reasons, deleted children, same numeric ID
across types, assigned IDs, complete ledger overrides and concurrent saves sharing
one Context. Actual SQLite tests cover SQL/audit propagation, provider rollback,
readback failure/retry and masking. Native batch diagnostics distinguish the
batch call's `batchOutcome` from an individual member's possibly unknown
`executionOutcome`. Same-type graph inserts now use a validated
`MutationBatchRequest` and the optional `BatchMutationExecutor` capability.
Physical JDBC rows retain separate immutable trace bindings, including failure
and readback diagnostics; incompatible insert column layouts are grouped separately.
Root intent remains required even if children are annotated or logs are disabled.
Providers without the capability retain individual command execution.

Internal reverse-list attachment-key projection preserves an explicitly requested
nested forward load. It must not use the public scalar selection operation that
removes a same-named relation load. Native SQLite tests cover root/nested graphs,
window/probe plans and logging on/off; the example gate runs these tests too.
Java retains an ID-only reference when its forward query has no matching target:
non-loaded fields remain guarded by `TeaQLNotLoadedException`, while list
membership and independent counts survive. This is not a claim of null-valued
reference parity with other runtimes.

Bootstrap follows the same request and audit boundary. Call
`context.ensureSchema()` with the generated Runtime Module installed: providers
perform physical DDL, then generated Q and audited Mutation reconcile roots and
constants. The legacy Portable `ensureSchema(context, type)` and repository
`ensureInitData(context)` data-write APIs have been removed. The
[School example](examples/school-management/README.md) verifies real bootstrap
SQL intent, committed lineage, fixed IDs, no-op reseeding and versioned constant
reconciliation on two starts of the same database.

The generated [Trace Chain example](examples/trace-chain/README.md) proves the
normative graph, overlapping three-level Q/E queries, late-consumed streams,
nested Facets with the original root and complete relation paths,
prepared insert grouping and complete ledger replacement, plus prepared
update/delete/recover batches with independent
optimistic versions. It now also runs real overlapping generated Checkers and
independent graph saves with one Context, observing per-item SQL and committed
audit lineage. Temporary check results, visited objects, Fix evidence and the
captured graph clock belong to each synchronous Checker invocation. Nested saves
restore the outer invocation while preserving the original custom Context and
its service hooks. `lastFixEvidence()` is the last completed check's diagnostic
receipt on the calling execution thread; it is not an async propagation API.
Read-only loaded relations retain their private ledger when reused by independent
graphs. Graph composition imports only pending mutations of explicitly visited
related entity keys, not every pending key from a foreign reference's ledger.
The provider-route guard also belongs to each mutation plan, not a retained
Context attribute. Independent graphs may use different providers on one
Context. A single atomic graph with writes to different routes is rejected
before mutation execution; read-only references do not count as writes.
Native tests cover separate SQLite databases and actual overlapping threads.

Native dynamic-aggregation tests also retain the original root through nested
relations, preserve inherited masking provenance and avoid fabricated relation
nodes for numeric partitions. They are separate from generated Facet acceptance.

Complete entry-point/privacy coverage, legacy unbound SQL diagnostic migration, asynchronous
handoff/cancellation and immutable internal Registry replay remain separate open
gates. The tested SQLite writer transactions serialize while the generated
Checkers overlap. This is local source evidence, not a merge or release claim.

Applications can replace runtime services such as `QueryPolicy`, the
`MutationPolicyRegistry`, `MutationPolicyApprovalProvider`, `RuntimeLogSink`,
`DataServiceRegistry`, `InternalIdGenerationService`, and `EntityMetaFactory`
in their integration layer.

Query and Mutation execution logs are enabled by default. The built-in default
sink is safe for ordinary operator output: it includes intent, trace, elapsed
time, outcome, and parameterized SQL, but excludes bind values and rendered
Debug SQL. Enable copy/paste SQL only for a controlled troubleshooting surface:
time, outcome, and SQL with safely rendered parameters. Sensitive values follow
the field's masking policy; an unsafe statement is omitted with a reason rather
than printed as plaintext. Select an additional diagnostic destination only for
controlled troubleshooting:

```java
TeaQLRuntime runtime = TeaQLRuntime.builder()
.metadata(metadata)
.queryExecutionLogging(true)
.mutationExecutionLogging(true)
.diagnosticSqlLogging(true) // values and Debug SQL; apply restricted retention
.diagnosticSqlLogging(true) // selecting a destination alone does not authorize plaintext
.build();
```

The Query and Mutation switches remain independent. Selecting diagnostic SQL
changes the built-in destination; it does not enable or disable either family.
Custom `RuntimeLogSink` implementations receive only parameterized SQL unless
they explicitly override `requiresSensitiveSqlData()` to return `true`.
Custom `RuntimeLogSink` implementations receive safe SQL projections too.
Plaintext requires both an explicitly sensitive destination and the exact
`TEAQL_ALLOW_SENSITIVE_PLAINTEXT_LOGS=I_UNDERSTAND_SENSITIVE_DATA_MAY_BE_WRITTEN_TO_DISK`
acknowledgement. Debug records are individually labeled; credentials remain protected.
Custom `UserContext` implementations must also explicitly delegate or override
`requiresSensitiveSqlLogData()` when they enable a diagnostic sink.
The optional file-backed `LogManager` requests value-bearing SQL only with
Expand All @@ -143,9 +244,9 @@ The optional file-backed `LogManager` requests value-bearing SQL only with
TeaQL Java is a server-side security reference runtime:

- ordinary Query and Mutation logs are enabled by default and retain intent,
trace, parameterized SQL, timing, and outcome without bind values;
- copy/paste SQL and parameter values require an explicitly selected sensitive
diagnostic sink;
typed trace, safely expanded SQL, timing, and outcome;
- ordinary SQL remains copy/paste-readable with masked values; plaintext requires
a sensitive diagnostic sink and the exact environment acknowledgement;
- the TFP endpoint applies trusted server policy, bounded queries, writable-field
rules, tenant scope, and optimistic version in the provider operation;
- boundary-facing entity references can be issued and verified through
Expand Down
26 changes: 21 additions & 5 deletions examples/school-management/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,15 @@

This generated example retains `models/school-model.xml`. It explicitly calls
SQLite `ensureSchema` twice and verifies Platform `id=1` plus SchoolType constants
`1001`/`1002` are present exactly once with version 1.
`1001`/`1002` are present exactly once. Fresh rows start at version 1.

The application-owned `BootstrapTraceVerifier` observes the generated bootstrap's
real SQL intent and committed audit lineage, without injecting trace frames.
It changes PRIMARY's name through audited Mutation, then calls ensureSchema to
restore the model-defined name. Each edit/repair advances the version once;
SECONDARY remains unchanged. Repeated ensureSchema performs lookups only, emits
no mutation audit, and restores the caller's bootstrap audit attributes. Both
fresh and already-seeded databases are exercised by the two-start verifier.

The application-owned `SchoolLifecycleVerifier` also checks a missing required
name is rejected by Checker before mutation or schema SQL during `save`,
Expand All @@ -11,16 +19,24 @@ loaded E traversal, full-field update, mark-for-deletion plus save, and normal-q
absence. The example gate runs with a fresh SQLite database; running it a
second time against the same database is supported.

`RequestIntentVerifier` proves generated Q requests and graph saves reject
missing comment/purpose before policy or SQL. It covers Unicode whitespace,
fabricated ambient intent and root intent inheritance through both forward
relations. Unit tests separately prove the gate is independent of logging.

For updates, load the complete scalar entity. A read projection that includes
only selected fields of related entities is useful for E/display, but should
not be reused as the mutation graph: Checker correctly rejects those partial
related entities as `NotLoaded`.

Before publication, install the repository's local runtime and then run the
generated workspace. The portable SQL runtime test separately changes a constant
and verifies optimistic, single-version reconciliation.
generated workspace. Portable provider tests forbid the removed raw seed APIs
and prove physical DDL never inserts/reconciles root or constant data. Typed
constant reconciliation is verified here, through the same generated API as an
application, rather than through a lower-level raw SQL shortcut.

From the repository root, run `examples/verify-runtime-examples.sh`. The gate
builds both retained examples against the current reactor sources, assigns each
run an isolated temporary SQLite database, waits for its acceptance marker, and
exits non-zero if either application fails or times out.
example an isolated temporary SQLite database, runs each twice without cleanup
between repetitions, and requires both the lifecycle and request-intent markers.
It exits non-zero if either application fails or times out.
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ public CommandLineRunner teaQLConsoleStartup(
if (!(dataServiceExecutor instanceof SchemaExecutor schema)) {
throw new IllegalStateException("default data service has no schema capability");
}
BootstrapTraceVerifier.verify(runtime);
context.ensureSchema();
context.ensureSchema();
SmartList<Platform> platforms = Q.platforms()
Expand All @@ -104,7 +105,8 @@ public CommandLineRunner teaQLConsoleStartup(
&& constants.get(0).getId() == 1001L
&& constants.get(1).getId() == 1002L,
"SchoolType constants were not seeded");
require(constants.get(0).getVersion() == 1L && constants.get(1).getVersion() == 1L,
// The bootstrap verifier made one audited edit and one audited repair to PRIMARY.
require(constants.get(0).getVersion() >= 3L && constants.get(1).getVersion() == 1L,
"Repeated ensureSchema was not idempotent");
require(new IdSpaceIdGenerator(database).nextId("SchoolType") > 1002L,
"SchoolType ID floor did not advance beyond model constants");
Expand Down
Loading
Loading