Skip to content

docs(skills): objectstack-query factual sweep — 17 false behavioral claims corrected, -33 lines - #13740

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-13717-skills-sweep-query
Aug 31, 2026
Merged

docs(skills): objectstack-query factual sweep — 17 false behavioral claims corrected, -33 lines#13740
os-zhuang merged 2 commits into
mainfrom
claude/issue-13717-skills-sweep-query

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #13717
Part of #13658

Flight ③ of the published-skills factual sweep: every behavioral claim in
skills/objectstack-query/** verified against the implementation, with executed
probes wherever a claim is behavior-bearing. Method per #13658 comments
5474435934 / 5474464789 / 5475032239 (content-class targeting first, both error
directions, cross-file contradiction scan first).

Governed surface — draft, human merge only. No ratchet ceiling raised.

Budget

before after delta
package lines (6 files) 1,550 1,517 −33
ratcheted tokens (5 files) 12,007 11,752 −255
SKILL.md 673 ln / 5,552 tok 658 ln / 5,443 tok −15 ln / −109 tok
rules/aggregation.md 278 / 2,357 264 / 2,226 −14 / −131
rules/filters.md 292 / 2,149 290 / 2,144 −2 / −5
rules/pagination.md 233 / 1,382 232 / 1,381 −1 / −1
evals/README.md 39 / 567 38 / 558 −1 / −9
references/_index.md 35 (generator-owned) 35 0

Every ceiling is shrink-only and every file came in under it. references/_index.md
was verified (all nine schema paths resolve at origin/main) and needed no edit.

Corrections — 落点 | before | after

17 FALSE claims corrected. 11 of the 17 are the deader-than-real direction — the
docs marked a shipped capability dead or unreachable. That matches flight ②'s
majority finding.

# 落点 before (claimed) after (measured)
1 SKILL.md Field References $field "schema-reserved — NOT executed… binds as a literal value, so the query silently returns zero rows" ✅ Enforced. In-memory evaluator resolves the reference; driver-sql pushes it down. Comparison position only — $in/$nin member and $between endpoint refused at parse
2 rules/filters.md Field References same claim, plus two "working alternatives" for a working feature same correction; alternatives deleted
3 SKILL.md aggregation driver-support callout SQL driver "throws on count_distinct"; "stick to the first five" ✅ All six portable — COUNT(DISTINCT x) on every SQL face; declared-but-uncompiled set is empty
4 rules/aggregation.md same callout same, quoting the error text Unsupported aggregate function same correction; that error string has zero occurrences repo-wide
5 SKILL.md aggregation callout per-aggregation distinct: true "is also ignored there" removed in 17, refused at parse — not ignored
6 rules/aggregation.md same same same
7 SKILL.md Filtered Aggregation per-aggregation filter "schema-reserved… returns the unfiltered number — silently wrong results" ✅ Enforced since the contract half of the 2026-08-21 ruling; forces the in-memory path; unknown operator inside it refuses INVALID_FILTER/400
8 rules/aggregation.md Filtered Aggregation same, with active_count would equal total! same correction
9 SKILL.md Dashboard Aggregation Pattern "a KPI with its own condition needs a separate call" one call, conditional measures carry their own filter
10 rules/aggregation.md DISTINCT Aggregation "Not available on SQL datasources"; and an example using the tombstoned distinct: true flag presented as a working alternative count_distinct runs everywhere; the non-parsing example deleted
11 rules/pagination.md DISTINCT Queries top-level distinct "schema-reserved… the query returns duplicate rows as if the flag were absent" ⛔ REMOVED in 17 — tombstoned, the query is refused at parse; QueryBuilder.distinct() gone
12 SKILL.md Full-Text Search prose "each term becomes an $or of $contains predicates… Matching is case-insensitive" $icontains$contains is contractually case-SENSITIVE, so the old text was self-contradictory
13 SKILL.md two asserted expansion outputs { title: { $contains: … } }; single-term wrapped in $and $icontains; single term is a bare $or, no $and wrapper
14 SKILL.md formula-mirror paragraph "Nothing rejects the mistake for you… a formula entry clears both lint and the ingress gate" both guards exist: a lint rule errors on a virtual searchableFields entry, and the ingress gate has a dedicated refusal branch for it
15 SKILL.md CRM blueprint, KPI row compareTo: 'previousPeriod' | 'previousYear' (bare string) compareTo: { kind: … } — the bare string was removed in 17 with no acceptance window
16 SKILL.md CRM blueprint, time-series row categoryGranularity pre-ADR-0021 removed key; dateGranularity on the widget's dataset selection
17 SKILL.md CRM blueprint, matrix row groupingsDown + groupingsAcross removed with the ADR-0021 cutover; dataset-bound rows + columns

Plus derived-claim repairs in evals/README.md (items 4, 5, 6, 7) whose rubrics
encoded the same falsehoods — item 6 told the model to fail an answer using
having, which is enforced.

One further correction, a framing rather than a surface claim: rules/filters.md
labelled { deleted_at: null } a ❌ Wrong null check. Measured, a bare null
comparand lowers to IS NULL on both the in-memory evaluator and the SQL driver
and returns exactly the same rows as $null: true. The recommendation to prefer
$null is kept (it is the only spelling of IS NOT NULL); the false "wrong"
label is not.

Cross-file contradiction scan (run first, per the method)

Four contradictions inside the package; every one settled against the
implementation, not against the other document:

  1. having — SKILL.md and rules/aggregation.md say ✅ Enforced; evals/README.md
    said "schema-reserved, silently dropped". Implementation: enforced.
  2. compareTo — SKILL.md said bare string, rules/aggregation.md says
    { kind, dimension? }. Implementation: the object form; the string is retired.
  3. windowFunctions — SKILL.md and rules/aggregation.md say REMOVED/tombstoned;
    evals/README.md said "schema-reserved". Implementation: removed.
  4. Per-aggregation filter — three files agreed it was inert; all three were wrong.
    (Agreement between documents is not evidence — this is the case the method's
    "never verify one document against another" rule exists for.)

Evidence — executed probes

Non-vacuity control satisfied several times over; every probe below ran a control
that discriminates.

  • $field cross-field. In-memory evaluator over three rows (over 200/100,
    under 50/100, equal 100/100): $gt["over"], $eq["equal"],
    $lt["under"]. Control, a plain literal $gt: 150 on the same harness →
    ["over"]. Through SqlDriver on better-sqlite3, same fixture pushed down:
    $gt { $field: 'budget' }["over"]. The claim under test predicted zero rows
    on both paths.
  • count_distinct on SQL. SqlDriver.aggregate with
    { function: 'count_distinct', field: 'dept', alias: 'n' }[{"n":2}].
    Control, an undeclared function median on the same driver → threw
    INVALID_QUERY/400, so a refusal would have been visible.
  • Per-aggregation filter. engine.aggregate('deal', …) over 4 rows
    (2 closed_won): [{"total_deals":4,"won_deals":2}]. The claim under test
    predicted won_deals: 4.
  • having. 4 orders across 3 customers, having: { order_count: { $gt: 1 } }
    [{"customer_id":"c1","order_count":2}]. Also confirms groupBy auto-selects
    the grouped column without listing it in fields.
  • Search expansion. expandSearchToFilter('machine learning', …)
    {"$and":[{"$or":[{"title":{"$icontains":"machine"}},{"content":{"$icontains":"machine"}}]},{"$or":[…"learning"…]}]};
    single term 'apollo' → a bare {"$or":[…]}.
  • Case sensitivity. Rows Apollo/apollo: $contains: 'apollo'
    ["apollo"]; $icontains: 'apollo'["Apollo","apollo"].
  • Parse oracle, tombstones. QuerySchema.safeParse REJECTS cursor, joins,
    windowFunctions, distinct, aggregations[].distinct, array_agg,
    string_agg, and an aggregation with no alias; ACCEPTS all six aggregation
    functions, top, having, aggregations[].filter, the fifteen documented filter
    operators, and a $field comparand.
  • Parse oracle, analytics. DashboardWidgetSchema REJECTS
    compareTo: 'previousPeriod' and categoryGranularity, ACCEPTS
    compareTo: { kind: 'previousPeriod' }; ReportSchema REJECTS
    groupingsDown/groupingsAcross, ACCEPTS dataset + rows + columns + values.
  • Token vocabulary (all VERIFIED). isDateMacroToken LIVE for today,
    yesterday, tomorrow, now, current_month_start, last_quarter_end,
    next_year_start, month_start, 30_days_ago, 2_weeks_from_now,
    current_year_start, current_year_end; isContextToken LIVE for
    current_user_id, current_org_id; both DEAD for the two near-misses the doc
    names as near-misses (current_user, this_quarter_start). Parameterised forms
    live for all six declared units in both directions.
  • $exists / $null (prior art of docs(skills): $exists means HAS A VALUE — retire the MongoDB $exists claim from objectstack-query #13577, re-verified at origin/main).
    Rows valued / null / key-absent: $exists: true["v"]; $null: true
    ["n","absent"]. HAS A VALUE, confirmed; both rows correct as merged.
  • Null comparand. { d: null }["n","absent"] in memory and ["n"] on
    SQL — identical to { d: { $null: true } } on each path. $null: false["v"].

Verified without execution, by enumerating the implementation: MAX_EXPAND_DEPTH = 3;
resolveFilterTokens reached from exactly find/findOne/count/aggregate/
update/delete; SqlDriver.findWithWindowFunctions() exists and its builder
emits argument-less FUNC() (its own docblock says the skills' aggregation rules
say the same); the six search knobs each carry an [EXPERIMENTAL — not enforced]
.describe() marker; the $searchFields dotted-path refusal text quoted in
SKILL.md matches the ingress gate word for word; the flow-filter rule really is
severity: inFilter ? 'error' : 'warning'; resolveDateMacros / resolveContextTokens
exist in objectui @object-ui/core; all four Skill Boundaries targets exist;
compatibility: @objectstack/spec 17.x (Zod v4) matches 17.2.0 on zod ^4.4.3.

Gates

Derived from the real diff with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
in the worktree, on a NON-stale tree (the branch was merged up to origin/main
0f911eb9 first — the first derivation warned that lint.yml and
cross-package-test-inputs.mjs, which are themselves families in the list, were
stale). 13 families, all run at d601906b, the final commit:

gate exit
node scripts/check-ci-filter-parity.mjs 0
node scripts/check-cross-package-test-inputs.mjs 0
node scripts/check-shard-attestation.mjs 0
node scripts/check-skills-token-ratchet.mjs 0
node scripts/check-test-completeness.mjs 3 — NOT MEASURED
pnpm --filter @objectstack/lint run check:doc-formula-expressions 0
pnpm check:agent-test-spelling 0
pnpm check:cross-package-test-inputs 0
pnpm check:doc-authoring 0
pnpm check:pm-governed-merges 0
pnpm check:role-word 0
pnpm check:skill-compatibility 0
pnpm check:skill-frame-sync 0

Exit codes captured with cmd > log 2>&1; EXIT=$? — before any pipe.

check-test-completeness exit 3 is the script's own PREREQUISITE NOT MET branch:
it grades a saved turbo run test log, and the derived family names it with no
argument. Its output says so verbatim and instructs the local reader to record it
as NOT MEASURED. Not a finding.

check:skill-examples — named in the dispatch brief, not in the derived family
list, and the derivation is right: that gate type-checks only fences marked with an
os:check HTML-comment marker, and skills/objectstack-query/** carries zero
such markers (control: objectstack-ui, objectstack-i18n and objectstack-formula
each carry them, so the search discriminates). Its population here is empty. Two
bounded foreground attempts to run it anyway were blocked by its
@objectstack/client-react build prerequisite, whose closure fails on pre-existing
type-only import gaps in @objectstack/runtime and @objectstack/verify
(Cannot find module '@objectstack/rest') — unrelated to this diff, and both
attempts hit the container's foreground cap. CI runs the gate on every PR regardless.

No changeset

Pure skills/** diff, releasing nothing. Convention verified against origin/main
rather than assumed: the last six commits touching skills/** — the three
objectstack-data sweep PRs, the objectstack-formula sweep, the $exists correction,
and the pm-dispatch rules PR — carry zero .changeset/ files between them. The
skip-changeset label is applied to this PR.

NOT MEASURABLE (recorded, never annotated into the files)

Nine claims could not be settled from a package-scoped flight; none was corrected
and none is known false. They are the cross-package-runtime class flight ① named:

  1. Expand resolves via batch $in queries "not N+1" — the QueryAST.expand
    .describe() asserts it and MAX_EXPAND_DEPTH = 3 is read directly, but proving
    the batching (rather than reading the claim) needs a driver call-count harness
    over a relational fixture.
  2. Expand applies fields + where only, ignoring per-parent limit/offset/
    orderBy — same fixture gap.
  3. $icontains ASCII-folding domain, and the __search pinyin companion column
    behind OS_SEARCH_PINYIN_ENABLED — needs a provisioned companion column.
  4. search fields narrowing answering 400 INVALID_FIELD over the REST/protocol
    ingress
    — the refusal text was matched in metadata-protocol, but the HTTP
    status pairing needs a live server.
  5. select/status label→value mapping in search — the unit suite pins
    { industry: { $in: ['retail'] } }; not re-run here against a real object.
  6. Date-bucket push-down vs in-memory fallback agreeing "including the column keys"
    — the alias-renames-the-projection half was measured
    ([{"quarter":"2025-Q1","revenue":30},{"quarter":"2025-Q2","revenue":5}]), the
    two-tier agreement across a real DATE_TRUNC dialect was not.
  7. ISO-8601 weeks starting Monday — asserted by DateGranularity's docblock; not
    executed across a week boundary.
  8. os validate / os build / npm run validate failing a dashboard widget whose
    dataset/dimensions/values do not resolve (ADR-0021) — needs a scaffolded
    project.
  9. Offset-pagination performance claims ("degrades on large offsets", keyset "O(1)")
    — true of relational engines generally; not benchmarked here, and not the kind of
    claim this sweep's oracle can settle.

Out-of-scope finding

packages/objectql/src/engine.ts carries a stale docblock describing the ADR-0061
search expansion as "a server-resolved cross-field $or of $contains" — the same
sentence this PR corrects in the published skill. The implementation next door
(search-filter.ts) emits $icontains and documents the correction explicitly. An
internal comment, outside this PR's file surface; reported for filing rather than
fixed here.

Generated by Claude Code


Generated by Claude Code

claude added 2 commits August 31, 2026 08:42
…laims corrected

Flight (3) of the published-skills factual sweep. Every behavioral claim in
skills/objectstack-query/** verified against the implementation with executed
probes; corrections are byte-shrinking against the pinned token ratchets.

Net: -33 lines, -255 tokens across the package. No ceiling raised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation labels Aug 31, 2026
@huangyiirene huangyiirene added skip-changeset PR has no user-facing published change; bypasses the changeset gate needs:contract-review and removed documentation Improvements or additions to documentation labels Aug 31, 2026 — with Claude

Copy link
Copy Markdown
Collaborator Author

Contract review — PASS at head d601906b

Applicability: clause-② CONTENT limb per the 2026-08-31 批 #12 ruling on #13588 — this diff makes falsifiable operator/contract semantic claims in published skills/** (the ruling's criterion, not mere contract mentions). Carriers were attached on PR and card #13717 at review start (a few minutes after PR creation — the dispatch predated the seat's read of the ruling; recorded, not silent).

Reviewer qualification, machine-read this hour, not self-described: get_sessionexternal_metadata.last_served_model = claude-fable-5 = CONTRACT_REVIEW_TIER (scripts/pm/dispatch-gates.mjs:6276). The reviewing seat is the dispatching seat — permitted since the 2026-08-21 relaxation (「你自己就是 Fable,自己就可以审核」): the object under review is the opus implementer's contract increment, not the reviewer's own product.

Review: full PR diff and report read; five anchors independently re-measured on origin/main this hour, all corroborating the corrections —

  • $field is a first-class comparand: packages/spec/src/data/filter-comparand-type.ts:202–208, with tests accepting comparison position and rejecting $in membership — the "binds as a literal, zero rows" claim was false;
  • packages/objectql/src/search-filter.ts:23 verbatim: "The case-insensitive operator is $icontains, NOT $contains";
  • count_distinct exercised on driver-sql with discriminating controls (INVALID_QUERY on the no-field case);
  • packages/spec/src/data/query.zod.ts:218 tombstone: aggregations[].distinct removed in spec 17 (ADR-0049) — refused at parse, not ignored; count_distinct is the live spelling;
  • packages/spec/src/ui/component.zod.ts:2454: compareTo is the { kind: … } object shape.

Every correction tightens toward the enforced contract or removes a dead claim; no acceptance widening. Budget shrink-only (−33 lines / −255 tokens, no ceiling touched).

needs:contract-review cleared on this PR and on #13717 in the same stroke. Governed .md content: this PR remains DRAFT for the maintainer's merge (os-zhuang / hotlong requested); the review chain never enqueues a governed surface.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skills-sweep ③: objectstack-query (1,550 lines, 6 files) — behavioral-claim verification, content-class execution-first

3 participants