Skip to content

feat(dream): add reviewed Memory-to-Experience-to-Skill workflows - #1510

Merged
Teingi merged 15 commits into
oceanbase:masterfrom
Teingi:worktree4_0907
Sep 10, 2026
Merged

feat(dream): add reviewed Memory-to-Experience-to-Skill workflows#1510
Teingi merged 15 commits into
oceanbase:masterfrom
Teingi:worktree4_0907

Conversation

@Teingi

@Teingi Teingi commented Sep 8, 2026

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

Closes #1509.

RFC: English · 中文.

Rationale for this change

Related task evidence can be repeated across Memory entries and Experience revisions without a reliable way to revisit the exact inputs, deduplicate their underlying observations, and propose a reviewed improvement. Artifact Dreaming adds an asynchronous workflow for turning selected Memory evidence into Experience and approved Experience into a managed Skill.

What changes are included in this PR?

  • Implement refine_experience and derive_skill with durable runs, immutable input manifests, idempotency, leases, bounded generation, recovery fencing, and atomic Candidate/run commits.
  • Resolve exact Memory entry versions and Experience lineage, group repeated root evidence, and recheck source eligibility, current authorization, entry activation, and target revisions. Prompt configuration remains lineage only.
  • Preserve entry citations through Candidate revision and approval, expose provenance in Review Inbox, and reuse the existing Experience/Skill approval and package lifecycle.
  • Keep ordinary Review validation separate from Dream generation budgets. Experience revision chains remain reviewable, and deriving a Skill from an Experience with 32 Sources preserves its direct Artifact reference without flattening those Sources into the Candidate. Iterative validation still checks transitive authorization and Memory validity; ordered Memory head locks protect approval against deactivation. Explicit Memory citations retain their required Source dependencies.
  • Add HTTP, Python SDK, CLI, MCP, capability discovery, and generated integration operation tables, with bilingual RFC and usage documentation.
  • Preserve upstream Topic Memory behavior, Prompt publication restrictions, and the reorganized documentation paths.

Are there any user-facing changes?

  • New endpoints: POST and GET /v1/scopes/{scope_id}/dream, plus GET /v1/scopes/{scope_id}/dream/{run_id}. New work returns 202; replaying a terminal idempotent request returns 200.
  • Add pc_dream_runs and nullable memory_citations columns on pc_artifacts and pc_artifact_candidate_versions, with additive initialization/migration for SQLite and OceanBase. Existing data remains readable. Explicit MySQL transactions preserve atomicity even with session autocommit enabled.
  • Experience proposals and candidate revisions accept exact Memory citations. Omitting citations or setting them to null during revision retains them; an explicit empty array clears them. The Candidate's combined direct-reference limit remains 32.
  • Dream results require review before becoming committed Artifacts. Generation is available only in the all process role; split api/background roles do not accept new Dream work. Dream and Topic Memory currently enforce separate execution concurrency limits.
  • Supervisor integration, automatic input discovery, Profile refinement, cross-Scope evidence expansion, and automatic installation are outside this scope.

How was this change tested?

  • make check: passed, including dependency lock consistency, pre-commit hooks, and type checks.
  • make contract-test: 46 passed; generated Python and JavaScript contracts match OpenAPI.
  • Review, Profile, HTTP authorization, and Dashboard regression tests: 95 passed, 2 optional-backend tests skipped.
  • Three new regression scenarios on SQLite and isolated OceanBase databases: 6 passed. They exercise 12 successive Experience revisions, transitive Memory deactivation beyond Dream's depth budget, and Skill proposal/revision/approval from an Experience with 32 Sources, including permission revocation on an indirect Source. Dream's own depth budget remains enforced.
  • Eight affected Dream evidence scenarios on both backends: 16 passed. They cover exact provenance, review edits, authorization, deactivation during generation, transitive Memory checks, root deduplication, atomic rollback, and Prompt lineage.
  • After integrating upstream changes, Prompt API, MySQL schema, integration manifest, documentation contract, and packaging tests: 46 passed.
  • Real configured LLM over a local HTTP server on SQLite and an isolated OceanBase database: Memory → Experience candidate → approval → recall → Skill candidate → approval, idempotent replay, and rejection of deactivated entries passed. Memory extraction used a deterministic fixture. Each generated Artifact used one model call; temporary databases were removed. These checks establish workflow correctness, not comparative quality or cost improvements.
  • make docs-test: passed; 770 static pages and their internal links verified.
  • git diff --check: passed.

AI usage statement

OpenAI Codex assisted with research, RFC writing, implementation, review fixes, merge integration, test execution, and PR preparation. Validation results and limitations are recorded above; no credentials or local environment files are included.

@Teingi
Teingi marked this pull request as ready for review September 9, 2026 03:09
Comment thread src/powercontext/builtin/review/service.py Outdated
@Teingi

Teingi commented Sep 9, 2026

Copy link
Copy Markdown
Member Author
image

Comment thread src/powercontext/builtin/dream/service.py
@Zxf-xufeng

Copy link
Copy Markdown
Member

P2: The lineage resolver stops at any artifact that is not an Experience, marking it unresolved and returning no children. This makes downstream Skill lineage opaque to Review: for a Skill derived from an Experience that carries Memory citations, the approval-time evidence check never reaches the upstream Memory entries. I reproduced this by deriving a Skill, retiring its original Memory entry, then creating a Skill replacement candidate; the candidate still approved successfully. Please keep Dream input restrictions separate from Review lineage validation, or traverse the stored Skill lineage so approval rechecks transitive Memory validity and authorization. The relevant code is in src/powercontext/builtin/evidence/resolver.py, around _read.

@Teingi

Teingi commented Sep 10, 2026

Copy link
Copy Markdown
Member Author

@Zxf-xufeng Fixed the Skill lineage issue from your comment in ec9036f.

Review now follows stored local Artifact lineage through Skill revisions, Experience, and Memory, independently of Dream's supported input families. Propose, revise, and approve recheck upstream Memory availability and current read authorization. A failed check leaves the candidate state unchanged. Ordinary Review also retains its separate reference budgets.

Added six regression cases covering those three operations with either a retired Memory entry or revoked read access. They pass on SQLite and OceanBase, alongside the existing transitive-lineage and reference-budget regressions. The fixes remain covered after the latest master merge, and the real SQLite/OceanBase + LLM Dream workflows pass.

@Zxf-xufeng Zxf-xufeng left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Teingi
Teingi merged commit c201666 into oceanbase:master Sep 10, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(dream): refine Memory and Experience into reviewed artifacts

2 participants