Conversation
e803351 to
6ea6fbd
Compare
frf12
changed the title
frf12
changed the title
| input_type=input_type, | ||
| output_type=output_type, | ||
| builtin_version=f"powercontext.topic_memory.{name}.v1", | ||
| invariant_instructions=_COMMON_INVARIANTS |
There was a problem hiding this comment.
[P2] Preserve the stage-specific rules in custom mode
PromptDefinition.resolve() compiles custom prompts from invariant_instructions and the saved guidance, without default_instructions. Setting only a language preference therefore removes the Planner's complete-partition/candidate-grouping rules and the Reducer's exact evidence-union and full-coverage rules. These relationships are not expressed by the JSON schemas, but the runtime still rejects outputs that violate them. Please retain these requirements in each stage's invariant instructions so a language-only customization does not remove the operation contract.
There was a problem hiding this comment.
Accepted — fixed in 3defaa0.
Each Topic Memory stage's schema-inexpressible operation contract now lives in its invariant_instructions (planner complete-partition/candidate-grouping, reducer exact evidence-union/full covered_indices coverage, reconcile historical-identity preservation, probe/temporary evidence grounding, evolve historical-target rule). A language-only customization now keeps these requirements; Auto mode is unchanged. Regression: test_custom_prompts_retain_stage_operation_contracts.
| ("skill.generate", skill_generator, generated_skill), | ||
| ("handoff.generate", handoff_pipeline, generated_handoff), | ||
| *( | ||
| (f"topic_memory.{stage}", None, object()) |
There was a problem hiding this comment.
[P2] Register demonstration generators for the new prompt keys
These entries advertise all seven Topic Memory keys as supported, but _register_prompt_demonstrators() still registers only the existing generation keys. With a configured generation model, I reproduced 422 prompt_customization_unavailable from every new /prompts/{key}/demonstrations endpoint, despite the configuration endpoint reporting supported. Please register the seven keys with the demonstration generator as well; adding them to the capabilities and OpenAPI enums alone leaves the new endpoints unusable.
There was a problem hiding this comment.
Accepted — fixed in 3defaa0.
_register_prompt_demonstrators() now registers all seven topic_memory.* keys alongside the existing ones. The e2e test generates topic_memory.probe demonstrations through the HTTP API with a configured generation model; the reproduced 422 prompt_customization_unavailable is gone.
| ) | ||
| output = self.output_type.model_validate_json( | ||
| json.dumps(demonstration.expected_output), strict=True, extra="forbid" | ||
| json.dumps(demonstration.expected_output), strict=strict, extra="forbid" |
There was a problem hiding this comment.
[P2] Validate relationships within Topic Memory demonstrations
validate_demonstration() has no Topic Memory branches, so validating the input and output separately accepts impossible examples. Through the SQLite-backed HTTP API, all seven keys accepted invalid demonstrations with 201 and returned them in the effective configuration. Examples include citing evidence/probe IDs absent from the input and returning covered_indices=[99] for a single-input reduction. These examples are then compiled into subsequent prompts even though the processing pipeline rejects equivalent outputs. Please validate evidence references, planner assignments, and reduction/reconciliation relationships before saving or resolving these demonstrations.
There was a problem hiding this comment.
Accepted — fixed in 3defaa0 (plus a ty narrowing fix in d2a2efb).
validate_demonstration() gained per-stage Topic Memory branches: cited evidence/probe/proposal IDs must exist in the input, planner outputs must partition every probe with exact candidate grouping, reduction must cover every zero-based input position with the exact evidence union and the matching output kind, and reconciliation must reuse input proposal_ids without merging historical identities. Your covered_indices=[99] example is now a regression case, along with unknown-reference, out-of-range, wrong-kind, and identity-merge cases.
There was a problem hiding this comment.
Rechecked on d2a2efb3. The original invalid-reference and covered_indices=[99] cases are now rejected, but two P2 validator/runtime mismatches remain:
-
Planner —
validation.py:200–208: shared candidates can still be split across create items. If two probes both list candidatec1, an output containing two separate items with nocandidate_idpasses demonstration validation and can be saved through the HTTP API (201). The same grouping fails in the SQLite processing pipeline withinvalid_plan, leaving the cursor unchanged. Thecandidate_id is Noneshortcut skips this case, while the runtime checks grouping for every shared candidate. Please enforce that rule regardless of whether an item selects a target, and keep the Planner invariants aligned with it. -
Reconcile —
validation.py:260: a valid new binding to supplied history is rejected. If input proposalp1has no target andhistoricalsuppliesc1, returningp1withcandidate_id=c1is a valid reconciliation. The SQLite pipeline successfully publishes revision 2 and advances the cursor, but the same demonstration returns422 prompt_definition_incompatible. Requiring output targets to be a subset ofinput_targetsexcludes this supported update path. Please allow targets supplied invalue.historical, while preserving existing proposal/target assignments and rejecting duplicate targets.
Both cases were verified through the Prompt HTTP API and the full SQLite processing pipeline with controlled stage outputs. The seven demonstration endpoints now return 200, and the exact used Prompt references survive spawned-worker execution and database reopening.
There was a problem hiding this comment.
Both confirmed and fixed in be4cf0b — thank you for the precise repro cases.
-
Planner grouping: the validator now mirrors
_plan'sitem_by_probecheck — for every candidate listed by any probe, all probes sharing it must land in the same work item, regardless of whether any item selects it as its target. Selected targets must also be unique across items (the runtime'stargetsuniqueness check). Your split-create case is now rejected (test_planner_demonstrations_group_shared_candidatescovers it and the single-item pass), and the Planner invariant text now states the shared-candidate rule explicitly. -
Reconcile binding:
allowed_targetsis nowinput_targets | supplied historical slots, matching_validate_reconciliation'sallowed_targetsfrom the component's history. An unbound proposal binding to a supplied historical candidate is accepted (test_reconcile_demonstrations_may_bind_unbound_proposals_to_supplied_history), while changing an existing assignment, duplicate targets, unknown proposal refs, and evidence outside the input union stay rejected. The Reconcile invariant text now mentions the binding path.
Verified locally: prompt suites 73 + 79 passed (incl. HTTP e2e), CI-exact uv run --locked ty check on the pypi index passes, ruff check/format clean.
| limits=limits, | ||
| model_settings=settings, | ||
| name=name, | ||
| prompt_key=prompt_key, |
There was a problem hiding this comment.
[P2] Persist the exact custom Prompt references in Topic lineage
Passing prompt_key supplies inference trace metadata, but the selected Prompt references never reach TopicMemoryDraft.artifacts. In a real spawned-worker test with SQLite and a loopback model stub, two Scopes used their respective custom Probe/Global instructions, yet both published Topics had lineage.artifacts=[], including after reopening the database. Once a Prompt head changes, the Topic cannot identify the revision used to generate it. Please carry the exact custom Prompt references from the stages actually used into the published Artifact lineage, preserving the distinction between configuration inputs and Source evidence.
There was a problem hiding this comment.
Accepted — fixed in 3defaa0.
TopicMemoryProcessor now receives the exact custom Prompt refs selected at compose time (selection.artifact per stage), records which stages actually ran in this window (via the existing reserve wrapper), and appends those refs to the published TopicMemoryDraft.artifacts — after the revised-topic ref, so configuration inputs stay distinguishable from Source evidence. A real SQLite pipeline e2e asserts the published lineage contains the used Probe ref and does not contain a stage that never ran. The upstream revision must exist (lineage FK), which the test honors by creating the prompt artifact first.
- keep stage operation contracts in invariant instructions so custom guidance cannot remove planner/reducer/reconcile relationships - register demonstration generators for the seven Topic Memory keys - validate cross-field relationships inside Topic Memory demonstrations - carry the exact custom Prompt references into published Topic lineage
…untime - reject plans that split probes sharing a candidate across items even when no item selects that candidate, matching the pipeline's invalid_plan check; selected targets must also stay unique - allow a reconcile demonstration to bind an unbound proposal to a candidate supplied in historical, matching _validate_reconciliation via allowed_targets = input targets | supplied history - extend the planner/reconcile invariant instructions accordingly
2 participants
Which issue or RFC does this PR close?
Closes #1532.
Rationale for this change
Topic Memory generation needs Scope-owned prompt configuration so each Scope can select the built-in prompt or provide a validated custom prompt without changing server-owned schemas or crossing Scope boundaries.
What changes are included in this PR?
Are there any user-facing changes?
The HTTP API exposes Scope-level Topic Memory prompt configuration and reports the effective built-in or custom prompt. No Dashboard or Profile changes are included.
How was this change tested?
uv run --no-sync pytest tests/builtin/artifacts/prompt/test_prompt_validation.py -q: 29 passed.uv run --no-sync pytest tests/e2e/test_prompt_management_api.py -q: 6 passed.git diff --check: passed.AI usage statement
OpenAI Codex assisted with implementation, tests and API verification.