Skip to content

fix(identity): stabilize idless message revisions (#3898) - #3898

Merged
Sinity merged 19 commits into
masterfrom
feature/fix/positional-message-identity-roundout
Aug 9, 2026
Merged

fix(identity): stabilize idless message revisions (#3898)#3898
Sinity merged 19 commits into
masterfrom
feature/fix/positional-message-identity-roundout

Conversation

@Sinity

@Sinity Sinity commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

Repair the exact-head identity and attachment-owner findings for PR #3898. The branch remains non-draft and is not being merged.

Problem

The exact review identified remaining collision and stability gaps: same-role same-timestamp idless siblings could be treated as conflicting on edit; whitespace native IDs diverged from missing IDs; duplicate physical coordinates could silently select a last writer; Claude owner evidence could omit attachment identity; and synthetic parser IDs could leak into public identity. The review also identified that the annotation successor's inventory must include annotation-batch:<id> rows without destroying that provenance.

Solution

  • Separate the timestamped idless revision axis from mutable message content and compare its multiplicity during membership classification.
  • Normalize whitespace native IDs before choosing the native-ID axis.
  • Make owner resolution fail closed for duplicate physical coordinates, including duplicate native IDs at one coordinate, instead of relying on map overwrite order.
  • Keep Claude attachment evidence in the private reorder-stable owner key even when explicit position evidence exists.
  • Keep synthetic public IDs empty on the exact Codex reasoning and compaction paths named by the review; native Codex IDs remain unchanged. Established synthetic conventions in other parser families were not changed after their existing public-contract tests rejected a provider-wide sweep.
  • Expand the durable annotation-owner successor contract to inventory both unscoped rows and annotation-batch:<id> rows, preserve the original batch scope, and store session ownership separately. No production backfill is performed here.

Acceptance and review matrix

Finding Disposition Evidence
Timestamped idless sibling edits collide or become contradictory satisfied SessionRevisionProjection.mutable_message_identities, multiplicity-aware membership, and test_timestamped_idless_sibling_edit_is_not_a_membership_conflict
Attachment-owner evidence is unstable or incomplete satisfied Claude private owner key includes attachment IDs; parser-to-writer and repeated-position anti-vacuity tests cover ownership
Duplicate physical coordinates or duplicate native IDs last-win satisfied MessageOwnerResolution records ambiguous physical coordinates and raises MessageOwnerAmbiguityError; both duplicate-coordinate tests exercise hash-route failure
Whitespace native IDs diverge from missing IDs satisfied _message_revision_match_id strips before branching; whitespace/missing projection equivalence test and writer regression pass
Synthetic public IDs leak from parser messages satisfied within named Codex scope Codex reasoning and compaction messages now retain empty provider IDs; parser tests prove the public field is empty and active-leaf resolution stays native-only
Annotation-batch owner scope partial, retained outside this PR successor polylogue-message-owner-scope-backfill is updated in the branch state with batch-row inventory, provenance preservation, separate durable session ownership, backup, transaction, idempotency, and reindex-preparation acceptance criteria

Bead disposition matrix

Assigned Bead Whole-Bead disposition Named successor
polylogue-slshy partial polylogue-message-owner-scope-backfill, polylogue-93xe, polylogue-xselt

Verification

  • direnv exec . devtools test tests/unit/pipeline/test_message_identity_position_fallback.py tests/unit/archive/test_session_revision_membership.py tests/unit/sources/test_claude_web_normalization.py tests/unit/sources/test_parsers_codex.py tests/unit/sources/parsers/test_grok.py tests/unit/sources/parsers/test_antigravity.py tests/unit/sources/test_parsers_claude_design.py tests/unit/pipeline/test_archive_write.py::test_whitespace_only_native_message_id_falls_back_and_writes_blocks tests/unit/pipeline/test_archive_write.py::test_duplicate_message_coordinates_raise_loud_value_error tests/unit/pipeline/test_archive_write.py::test_merge_append_duplicate_message_coordinates_also_guarded: 218 passed in 10.59s.
  • direnv exec . devtools test tests/unit/pipeline/test_message_identity_position_fallback.py: 10 passed in 1.10s after the duplicate-native anti-vacuity addition.
  • direnv exec . devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/parsers/test_grok.py tests/unit/sources/parsers/test_antigravity.py tests/unit/sources/parsers/test_parsers_claude_design.py was run with the repository's actual path as tests/unit/sources/test_parsers_claude_design.py: 147 passed in 7.62s.
  • direnv exec . devtools verify --quick: success at 1eb36fdb171210ce6fc5440e6fef621605f5a58c, 24 steps passed, including format, lint, mypy, render, layering, policy, schema, and promotion checks.
  • direnv exec . devtools verify: attempted, but refused before tests because this aging worktree has no seeded testmon receipt and requested --seed-testmon.
  • direnv exec . devtools verify --all --skip-slow: attempted as a diagnostic and stopped after broad inherited failures; the exact focused repair routes above are the completion evidence. The known untouched baseline failure is tests/unit/pipeline/test_archive_write.py::TestValidationService::test_validation_uses_all_record_samples_by_default.
  • No Beads command, durable backfill, source/user-tier mutation, index reset, semantic reindex, or merge was performed.

Review status

All unresolved GitHub review threads were enumerated. Addressed threads are replied to with exact evidence; the annotation-batch threads retain the explicit partial successor scope. A fresh exact-head Codex review is requested after pushing this head. No merge is requested.

Structured exact-head carrier

Summary by CodeRabbit

  • Bug Fixes

    • Preserved marks and annotations across session-index rebuilds and legacy data migrations.
    • Improved session alias matching, including provider-qualified and native identifiers, while safely rejecting ambiguous matches.
    • Ensured message-targeted marks do not appear as session-level marks.
    • Improved attachment ownership when messages are reordered, duplicated, or lack provider IDs.
    • Prevented stale authority decisions from being treated as current.
  • Reliability

    • Improved handling of id-less and timestamped messages during reindexing and revision comparisons.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds stable ownership for id-less and duplicate messages, propagates durable session ownership for marks and annotations, updates attachment and revision handling, and migrates affected fixtures to canonical archive identifiers.

Changes

Identity and owner-scope repair

Layer / File(s) Summary
Stable message and attachment identity
polylogue/core/*, polylogue/pipeline/ids.py, polylogue/sources/parsers/*
Messages and attachments now use owner coordinates, stable evidence keys, canonical identities, and explicit ambiguity handling.
Revision and attachment ownership writes
polylogue/archive/*, polylogue/storage/attachment_relink.py, polylogue/storage/sqlite/archive_tiers/write.py
Mutable id-less identities affect revision comparison. Attachment writes and relinking use shared owner-key resolution.
Durable assertion owner scope
polylogue/api/archive.py, polylogue/operations/mutation_actuators.py, polylogue/storage/sqlite/archive_tiers/*
Mark and annotation writes persist owner sessions. Reads resolve owners from durable scope or indexed message rows and apply session filters.
Canonical identity and incident coverage
tests/*, polylogue/storage/raw_authority.py, .beads/issues.jsonl, docs/plans/*
Fixtures use canonical archive identifiers. Tests cover ownership and ambiguity cases. Authority fingerprints, web-shell filtering, and incident successor records were updated.

Estimated code review effort: 5 (Critical) | ~90 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.88% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning No pull request description was provided, so all required template sections and verification details are missing. Add the required Summary, Problem, Solution, Verification, Bead disposition matrix, Changelog, and Risks and Follow-ups sections.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the primary change: stabilizing identity for id-less message revisions.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/fix/positional-message-identity-roundout

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Sinity Sinity changed the title fix(identity): stabilize idless message revisions fix(identity): stabilize idless message revisions (#3898) Aug 8, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91a17b84b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +9332 to +9335
for marker in (":n:", ":p:"):
marker_index = target_id.find(marker)
if marker_index > 0:
return target_id[:marker_index]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve ownership without the first tag marker

When a provider session's native ID itself contains :n: or :p:—which the identity law permits because native IDs are opaque—find() stops at that embedded substring instead of the actual message-local delimiter. For example, a message ID such as codex-session:run:n:child:n:m1 is reported as belonging to codex-session:run, causing list_marks() and list_annotations() to return an incorrect session_id. Resolve the owner through the indexed message/session relation or another unambiguous representation rather than the first textual marker.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at exact head 1eb36fdb171210ce6fc5440e6fef621605f5a58c. The target reference is split only at the message: prefix, so embedded :n: or :p: tokens in the opaque message ID remain part of target_id. The earlier implementation concern is outdated; the current indexed lookup uses the complete message ID.

Comment thread .beads/issues.jsonl Outdated
@@ -183,7 +183,7 @@
{"_type":"issue","id":"polylogue-a7xr.24","title":"Spec-driven row↔domain hydration: one declaration drives DDL, records, mappers, hydrators","description":"Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-dissection-2026-08-03.html; ledger .agent/scratch/dissection-ledger.md) traced one message field (stop_reason/is_active_leaf) vertically: ColumnSpec (archive_tiers_specs.py) already claims 'single source of truth driving INSERT/SELECT' yet the messages DDL in index.py:539-610 is hand-written separately, and the mechanical middle restates the same shape four more times: storage/runtime/archive/records.py (462), queries/mappers_archive.py (319), storage/hydrators.py (258), semantic/facts.py protocol (~150 of 503), message_query_reads.py SELECT plumbing (~300 of 678) — ~1.5K lines for the message family alone, similar for sessions/blocks/attachments. Target: extend ColumnSpec (or a successor spec) so DDL, record dataclass, row mapper, and domain hydration derive from one declaration; measured AC: a new message-family column reaches the domain model by editing the spec + lifecycle delta only (2 files below the surface boundary, vs ~8 today). Falsification: next real v-bump field's file-touch count below the api/ boundary. Fold-in: stop_reason is TWO unrelated concepts sharing a name (message stop_reason vs embedding-run stop_reason in cli/embed.py + storage/embeddings/progress.py) — rename one during adoption.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T13:14:47Z","created_by":"Sinity","updated_at":"2026-08-03T13:14:47Z","labels":["area:substrate","delivery:M-substrate-consolidation","horizon:mid","lane:substrate-consolidation","spine"],"dependencies":[{"issue_id":"polylogue-a7xr.24","depends_on_id":"polylogue-a7xr","type":"parent-child","created_at":"2026-08-03T15:14:47Z","created_by":"Sinity","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0,"metadata":{"acceptance_contract_v1":{"anti_vacuity":["A controlled mutation that removes the central guard or restores the pre-fix behavior makes the focused regression fail.","The test asserts durable/public behavior, not merely that a helper was called or returned a mocked value."],"bead_id":"polylogue-a7xr.24","closure":{"disposition":"whole-or-explicit-partial","rule":"Close `polylogue-a7xr.24` only when the criteria above are evidenced on the final head. Any residual operation, provider/origin, live population, or generalized bug class is transferred to a named successor with a dependency edge before closure.","successor_required_for_partial":true},"confidence":"high","contract_type":"implementation","evidence":["The regression or audit preserves the motivating observation: Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-dissection-2026-08-03.html; ledger .agent/scratch/dissection-ledger.md) traced one message field (stop_reason/is_active_leaf) vertically: ColumnSpec (archive_tiers_specs.py) already claims 'single source of truth driving INSERT/SELECT' yet the messages DDL in index.py:539-610 is hand-written separately, and the mechanical middle restates the same shape four more times: storage/runtime/archive/records.py (462), queries/mappers_archive.py","Measured evidence remains reconciled with this recorded population: DDL, records, mappers, hydrators Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-disse","Measured evidence remains reconciled with this recorded population: records, mappers, hydrators Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-dissecti"],"generated_at":"2026-08-07T00:00:00Z","outcome":"The production path no longer exhibits the defect or missing capability named “Spec-driven row↔domain hydration: one declaration drives DDL, records, mappers, hydrators”; the result is observable through the public or operator-facing route.","retained_scope":[],"risk":"ordinary","routes":["Exercise the implementation through these named production surfaces: `.agent/scratch/dissection-ledger.md`, `stop_reason/is_active_leaf`, `INSERT/SELECT`, `storage/runtime/archive/records.py`."],"safety":[],"schema_version":1,"source_digest":"6cf7c46cec513d99396532acfca48acc690f8a675b1ce65ad328c5d8336d45d8","verification":["Add a focused red-before/green-after regression carrying `polylogue-a7xr.24` or the incident name and executing the owning production route.","Run `devtools verify --quick` on the final head and record the exact head SHA in the closure evidence.","Run `devtools verify` on the final head so the testmon-affected regression set executes; `devtools verify --quick` alone is insufficient.","Run `devtools verify` on the final head so the testmon-affected regression set executes; `devtools verify --quick` alone is insufficient."]}},"acceptance_criteria":"1. Outcome: The production path no longer exhibits the defect or missing capability named “Spec-driven row↔domain hydration: one declaration drives DDL, records, mappers, hydrators”; the result is observable through the public or operator-facing route.\n2. Production route: Exercise the implementation through these named production surfaces: `.agent/scratch/dissection-ledger.md`, `stop_reason/is_active_leaf`, `INSERT/SELECT`, `storage/runtime/archive/records.py`.\n3. Evidence: The regression or audit preserves the motivating observation: Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-dissection-2026-08-03.html; ledger .agent/scratch/dissection-ledger.md) traced one message field (stop_reason/is_active_leaf) vertically: ColumnSpec (archive_tiers_specs.py) already claims 'single source of truth driving INSERT/SELECT' yet the messages DDL in index.py:539-610 is hand-written separately, and the mechanical middle restates the same shape four more times: storage/runtime/archive/records.py (462), queries/mappers_archive.py\n4. Evidence: Measured evidence remains reconciled with this recorded population: DDL, records, mappers, hydrators Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-disse\n5. Evidence: Measured evidence remains reconciled with this recorded population: records, mappers, hydrators Dissection 2026-08-03 (report: /realm/data/derived/reports/polylogue-structural-dissecti\n6. Verification: Add a focused red-before/green-after regression carrying `polylogue-a7xr.24` or the incident name and executing the owning production route.\n7. Verification: Run `devtools verify --quick` on the final head and record the exact head SHA in the closure evidence.\n8. Verification: Run `devtools verify` on the final head so the testmon-affected regression set executes; `devtools verify --quick` alone is insufficient.\n9. Verification: Run `devtools verify` on the final head so the testmon-affected regression set executes; `devtools verify --quick` alone is insufficient.\n10. Anti-vacuity: A controlled mutation that removes the central guard or restores the pre-fix behavior makes the focused regression fail.\n11. Anti-vacuity: The test asserts durable/public behavior, not merely that a helper was called or returned a mocked value.\n12. Closure disposition: whole-or-explicit-partial\n13. Closure: Close `polylogue-a7xr.24` only when the criteria above are evidenced on the final head. Any residual operation, provider/origin, live population, or generalized bug class is transferred to a named successor with a dependency edge before closure."}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update reindex coverage when closing the blocker

Closing polylogue-slshy here leaves both docs/plans/reindex-incident-coverage.json and tests/fixtures/reindex_incident_coverage/campaign_graph.json describing it as an in_progress blocking parser-stamps dependency with no receipt. resolve_default_incident_coverage() consumes those committed artifacts, so the reindex preflight cannot observe this completion; merely changing their status to closed would also trigger the validator's requirement for a live-proof receipt or named successor. Update the structured coverage records and evidence alongside this closure.

AGENTS.md reference: AGENTS.md:L294-L299

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head keeps polylogue-slshy explicitly in_progress and partial. docs/plans/reindex-incident-coverage.json records its named residual polylogue-message-owner-scope-backfill, and the campaign graph records both named child successors polylogue-xselt and polylogue-message-owner-scope-backfill. No closure or broad Beads mutation was made. The finding assumes a closed disposition and is stale against the current structured state.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This remains intentionally partial on the new head. The structured disposition and carrier preserve polylogue-slshy as in_progress with polylogue-message-owner-scope-backfill, polylogue-93xe, and polylogue-xselt named for residual scope. This repair adds the duplicate-owner hash and Claude private evidence-key coverage without changing that disposition or performing the durable backfill.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at exact head. The bead remains explicitly partial and in progress, with the owner-backfill successor named in both .beads/issues.jsonl and the exact-head carrier. No closure mutation was made, so the coverage preflight is not being asked to treat this bead as complete.

@Sinity

Sinity commented Aug 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74f32af738

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 9352 to 9355
# A message assertion can outlive its indexed message. Retain the
# historical delimiter fallback for those legacy/unresolved IDs.
session_id, _sep, _message_native_id = target_id.rpartition(":")
return session_id

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the owner when the indexed message is absent

When a durable message assertion outlives its rebuildable indexed row—for example after a hard delete or index reset—this fallback runs, but canonical message IDs are shaped like session_id:n:native_id or session_id:p:position.variant, so rpartition(":") returns session_id:n or session_id:p rather than the owning session (and becomes still less reliable when the native ID contains colons). Consequently list_marks() and list_annotations() expose the wrong session_id precisely while the indexed row is unavailable. Fresh evidence at this head is the newly explicit outlived-message fallback on these lines; preserve the owner in durable metadata or use another unambiguous representation instead.

AGENTS.md reference: AGENTS.md:L118-L122

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at exact head. The current reader does not guess an owner from a message ID when the indexed message is absent; it returns an unresolved owner and leaves durable ownership to the named backfill successor. This outdated fallback concern is therefore not a branch-local regression.

Comment thread .beads/issues.jsonl Outdated
{"_type":"issue","id":"polylogue-gzgyl","title":"material_origin HUMAN_AUTHORED regression: add positive-evidence overrides to 6 chat-product parsers before reindex (PR #2502 gap)","description":"Flagship finding of the 2026-08-03 reindex-gate-hunt (tasks #7 + #3 + #13; adjudicated P but practically must-fix-before-818fy: this is a REGRESSION the reparse would actively cause, not a static gap).\n\nPR #2502 (commit 7120bde61) removed the shared Role.USER+MESSAGE -> HUMAN_AUTHORED fallback from classify_material_origin (artifacts.py:191-199 now deliberately returns UNKNOWN — correct reasoning for agent runtimes). Only Codex (_codex_material_origin) and Claude Code (_claude_code_user_turn_origin) got compensating parser-level overrides. Every other origin classifies user turns UNKNOWN under current code.\n\nMeasured (live archive, 2026-08-03): gemini-cli-session already 100% regressed (66 unknown / 0 human_authored — parsed post-change, the smoking gun). Still-correct rows surviving ONLY via content-hash idempotency (would flip to unknown on full reparse): chatgpt-export 13,912; aistudio-drive 6,838; claude-ai-export 1,593; grok-export 13. Hermes split-proof (task #13): sessions via hermes_state.py state-db path 408/408 human_authored; via local_agent.py wire path 1105/1105 unknown — 100% clean split on the hermes:state-db session tag.\n\nFIX LIST (per-parser positive-evidence overrides; do NOT restore the generic fallback in artifacts.py — its no-fall-through reasoning stands for agent runtimes):\n- chatgpt.py (consumer chat export: role=user is positive human evidence)\n- claude/ai_parser.py (same)\n- gemini_message.py (AI-Studio/Drive)\n- local_agent.py: parse_gemini_cli AND _parse_hermes_message (~line 250) — two separate gap sites in one file\n- grok.py\n- antigravity.py (task #3: its \"User Input\" turns currently all UNKNOWN)\nMirror the Codex override pattern: UNKNOWN + Role.USER + MessageType.MESSAGE -> HUMAN_AUTHORED at parser level, scoped to each parser's genuine-user-turn shapes.\n\nWhy the blocks-818fy edge: unlike sibling P items, an unfixed reparse OVERWRITES ~22,400+ currently-correct rows with unknown, degrading the authored-user cost/word accounting axis (the exact axis CLAUDE.md calls load-bearing) across the largest chat-product origins.\n","acceptance_criteria":"Each listed parser sets material_origin=HUMAN_AUTHORED for genuine user turns via its own positive evidence. Live-fixture tests per origin assert role=user MESSAGE rows classify human_authored. artifacts.py classify_material_origin is NOT modified. Post-fix reparse of a sample session per origin shows zero human-turn unknown regressions vs current archive.","notes":"Fix-list verification (iteration 5, 2026-08-03, current source): gemini_message.py, local_agent.py, grok.py, antigravity.py contain ZERO material_origin handling (rg count 0 each) — all four need the positive-evidence HUMAN_AUTHORED override. chatgpt.py (2 refs) and claude/ai_parser.py (3 refs) have PARTIAL handling — verify at execution time whether their existing refs classify plain human messages or only edge shapes; extend rather than duplicate. artifacts.py's no-fall-through stays (correct for agent runtimes).\n\n2026-08-03 continuation: implemented the full fix list. Added shared polylogue/sources/parsers/base_support.py::human_authored_override(role, message_type, material_origin) helper (mirrors the Codex/Claude-Code override pattern) and wired it at every genuine plain-user-turn construction site across the 6 target parsers, PLUS one site the bead's fix list didn't name but that carries the exact same bug: claude/common.py::normalize_chat_messages (the ORDINARY claude.ai chat_messages bulk path -- ai_parser.py's own 2-3 refs are only the Claude Design / memories edge cases; the actual bulk of claude-ai-export's 1,593-row exposure lives in common.py, which had ZERO material_origin handling before this fix).\n\nSites fixed: chatgpt.py (1, already had classify_material_origin, wrapped with the override), claude/common.py::normalize_chat_messages (1, new -- the real bulk fix), claude/ai_parser.py (2: user_interjection + _design_user_message), local_agent.py (2: _parse_gemini_message, _parse_hermes_message), grok.py (1), antigravity.py (1), drive.py (1, AI-Studio/Drive).\n\nSelf-caught regression during verification: the naive first-pass wiring at drive.py/local_agent.py hardcoded MessageType.MESSAGE when calling classify_material_origin, without first resolving the block-derived message_type (classify_block_message_type) -- this silently broke TOOL_RESULT classification for messages whose blocks (codeExecutionResult, tool_calls) would have reclassified them, since my explicit material_origin= now bypasses the shared model_validator's own UNKNOWN-gated reclassification. Caught by test_ai_studio_normalizes_identity_authorship_config_blocks_artifacts_usage_and_status (assert ASSISTANT_AUTHORED != TOOL_RESULT) BEFORE committing. Fixed by resolving message_type from blocks first at both risk sites (drive.py, local_agent.py's two functions) before calling classify_material_origin -- chatgpt.py/claude/common.py were already safe (explicit message_type computed pre-existing; ai_parser.py/grok.py/antigravity.py sites never carry tool-shaped blocks, so the simpler form is correct there).\n\nUpdated tests/unit/sources/test_gemini_drive_normalization_laws.py's test_gemini_cli_schema_fields_survive_dispatch_without_export_authorship_upgrade (renamed to ..._with_human_authored_override) -- it asserted UNKNOWN as correct, documenting the pre-fix regression by name; now asserts HUMAN_AUTHORED.\n\nVerification in progress: devtools test tests/unit/sources/ -> 2327 passed / 2 known (4987i unrelated + the renamed test, now fixed). Broader tests/unit/sources+pipeline+storage sweep running.\n\nNOT YET: full verify --quick, commit, or the \"Fix-list verification... rg count 0\" style audit of whether MORE sites exist beyond what's covered (this session did not re-run the bead's own verification rg commands after the fix to confirm zero remaining zero-material_origin-handling parsers).\n\n2026-08-03 landed + pushed: commits e1db12a0c (fix), 1bd6940bd (demo-tour report.json fallout), 4fec7153b (demo-corpus-datasheet fallout, same root cause). devtools verify --quick clean. All 7 target sites fixed (6 from the bead's fix list + claude/common.py's normalize_chat_messages, the real bulk claude-ai-export path the fix list's ai_parser.py entry didn't actually cover). Full sources+pipeline+storage sweep: 5048 passed, 1 known unrelated (4987i).\n\nClosing -- fix list complete, verified live-shaped (block-derived-type regression caught and fixed pre-commit), fallout regenerated. Live-archive reparse impact (the ~22,400 rows this bead measured) will be realized when 818fy's actual reindex runs; no further code work needed from this bead.","status":"closed","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:18:21Z","created_by":"Sinity","updated_at":"2026-08-03T16:03:52Z","closed_at":"2026-08-03T16:03:52Z","close_reason":"All 7 sites (6 bead-listed + claude/common.py bulk path) fixed and verified. Commits e1db12a0c/1bd6940bd/4fec7153b.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"polylogue-s8s54","title":"Retroactive origin reclassification for 41 pre-fix browser-capture ChatGPT raws (mvq8 unmet AC2, S-class)","description":"S-class (durable-tier), adjudicated 2026-08-03 (reindex-gate-hunt task #14 Finding 2) as duplicate of polylogue-mvq8's UNMET AC(2) — successor bead carrying exactly that deferred scope.\n\nEvidence: 41 of 844 raw_sessions with source_path under /browser-capture/chatgpt/ carry origin=unknown-export (4.9%); the other 803 are chatgpt-export. Root cause CLOSED by blob-level diff (hunter-parser): NOT a live detector bug — all 41 acquired BEFORE PR #3558 (merged 2026-08-02, mvq8 AC1), which fixed the provider marker sitting past the old 1MiB prefix-scan window (raw_provider_payload sorts alphabetically before session.provider under sort_keys=True). Envelope shape byte-identical between groups; session.provider==\"chatgpt\" present and valid JSON in the misdetected blobs. New acquisitions classify correctly today.\n\nWhy S / why before 818fy: origin is stamped durably at acquisition into source.db and never re-derived; sessions.session_id is a GENERATED column origin||\":\"||native_id. The 818fy reparse would bake the wrong origin AND wrong session identity in again for these rows; fixing after means a second identity churn. Repair machinery already exists: polylogue/storage/unknown_export_reclassification.py (read-only classifier built for exactly this). Scope = pure repair run + receipt, NO parser change.\n\nOne of the 41 is also one of the 6 unmaterialized raws cited in the f1vg audit-trail note (task #14 Finding 1) — reclassification may unblock its materialization; verify while in the area.\n","acceptance_criteria":"The 41 raw_sessions rows under /browser-capture/chatgpt/ with origin=unknown-export are reclassified to chatgpt-export in source.db via an operator-authorized apply pass driven by storage/unknown_export_reclassification.py (or successor actuator). Post-repair query shows 0 unknown-export rows on that acquisition path. Repair receipt recorded. 818fy reparse then derives correct session_id identity for all 41.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:18:21Z","created_by":"Sinity","updated_at":"2026-08-03T12:18:21Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-04T08:00:51Z","started_at":"2026-08-04T08:00:51Z","lease_expires_at":"2026-08-04T08:05:51Z","heartbeat_at":"2026-08-04T08:00:51Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-08T22:07:22Z","started_at":"2026-08-04T08:00:51Z","closed_at":"2026-08-08T22:07:22Z","close_reason":"Completed positional message identity roundout: production comparison anchors no longer depend on array position, idless multiplicity and attachment ownership are preserved, canonical fixture constructors cover generated IDs, 147 focused tests passed, and all 24 quick gates passed in commits 15e134149 and 5a21b86cf.","dependency_count":0,"dependent_count":1,"comment_count":0}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run the default verification before closing the bead

This closure records only 147 focused tests and the 24 quick gates, even though the change alters production identity/storage behavior and the repository explicitly states that devtools verify --quick has no tests and is not a substitute for the default baseline. Closing the bead as completed without running and recording devtools verify leaves the durable task state claiming satisfaction without the required affected-test selection; run the default final-head verification, classify the acknowledged red test, and then record the resulting evidence.

AGENTS.md reference: AGENTS.md:L344-L346

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default gate was attempted at this repair head and refused before test execution because the aging worktree has no seeded testmon receipt. The bead remains partial, the exact focused repair routes and devtools verify --quick are recorded, and I am not claiming the default affected-test gate passed.

Sinity added a commit that referenced this pull request Aug 8, 2026
Problem: PR #3898 records durable ownership for new message assertions, but legacy unscoped assertions remain unresolved after index loss and the default affected-test receipt is unavailable while testmon is unseeded.

What changed: Reopen polylogue-slshy as active, add the typed backup-gated backfill successor, link the verification blocker polylogue-93xe, and align the reindex incident ledger and campaign graph with the partial state.

Compatibility/migration: This commit changes tracker and incident authority only. It performs no production backfill, index reset, or reindex.
@Sinity

Sinity commented Aug 8, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7921d08f11

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py Outdated
if message.timestamp:
return f"{_TIMESTAMP_ANCHOR_PREFIX}:{message.role}:{message.timestamp}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'text': _normalize_for_hash(message.text)})}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'timestamp': _normalize_for_hash(message.timestamp), 'text': _normalize_for_hash(message.text)})}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bump the revision-membership fingerprint

When an existing archive has durable parser-census or authority verdicts stamped revision-membership-v2, this new role/timestamp/text anchor changes the SessionRevisionProjection semantics without changing RAW_AUTHORITY_PARSER_FINGERPRINT or superseding v2. For example, two idless turns with the same role and timestamp previously shared an attachment-owner anchor but are now distinguished by text; any verdict computed under the old projection remains classified as current by require_current_parser_source_census() and can be reused during replay/reindex instead of being recomputed. Bump the fingerprint and mark v2 superseded alongside this semantic change.

AGENTS.md reference: AGENTS.md:L114-L122

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. The revision-membership fingerprint already covers the parser semantics, and the new projection keeps timestamped idless identity on the role/timestamp axis while content remains mutable. The sibling-edit membership regression passes against the real projection and membership relation.

Comment on lines +9387 to +9390
row = index_conn.execute(
"SELECT session_id FROM messages WHERE message_id = ?",
(target_id,),
).fetchone()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Batch indexed owner lookups before listing assertions

When a user has many message-target marks or annotations lacking the new durable scope, each result now executes its own SELECT against messages; list_annotations(session_id=...) executes it twice for every matching row. Because both list methods first load the complete assertion kind, this turns these public reads into O(N) SQLite statement executions and can make large user-state archives noticeably slow. Resolve legacy owners in one batched query or join instead of querying once per assertion.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. _user_state_session_ids batches unresolved message IDs in chunks of 500 and performs one indexed query per chunk; the old per-assertion query pattern is no longer present. The existing batch-lookup regression remains green.

Comment thread polylogue/pipeline/ids.py Outdated
if message.timestamp:
return f"{_TIMESTAMP_ANCHOR_PREFIX}:{message.role}:{message.timestamp}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'text': _normalize_for_hash(message.text)})}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'timestamp': _normalize_for_hash(message.timestamp), 'text': _normalize_for_hash(message.text)})}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include structured blocks in idless owner anchors

When two distinct idless messages have the same role, timestamp, and text but different structured blocks, this fallback still assigns them the same owner anchor. _session_hash_components() then substitutes that anchor for an idless attachment's message_id, so moving an attachment between those messages leaves both the attachment payload and revision identity unchanged; because the messages themselves did not change, session_content_hash() also matches and re-ingest skips the ownership update entirely. Fold the content-block payload into the idless anchor so structurally distinct messages cannot alias.

AGENTS.md reference: AGENTS.md:L136-L144

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Structured blocks are part of _message_comparison_payload, so duplicate idless messages with different structured blocks receive distinct content keys. The owner-resolution tests exercise the block-sensitive branch.

"target_type": found_target_type,
"target_id": found_target_id,
"session_id": _user_mark_session_id(found_target_type, found_target_id),
"session_id": owner_session_id,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep message marks out of session mark state

When a message-target mark is listed, this now emits the real owning session in session_id, but daemon/web_shell.py's loadUserState() treats every returned row as a session mark and calls setMarkLocal(m.session_id, m.mark_type, true) without checking target_type. Adding a pin or star to a message therefore makes the whole session appear pinned or starred, and removing the separate session-level mark only makes it reappear after reload. Filter non-session targets in that consumer or maintain target-scoped mark state.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. loadUserState() now ignores non-session marks before calling setMarkLocal, and the focused privacy contract test asserts the guard. Message marks remain target-aware in the returned state.

# a timestamp-only fallback would make moving the attachment invisible.
first: JSONDocument = {"role": "user", "text": "first", "createTime": "2026-01-01T00:00:00Z"}
second: JSONDocument = {"role": "model", "text": "second", "createTime": "2026-01-01T00:00:01Z"}
second: JSONDocument = {"role": "model", "text": "second", "createTime": "2026-01-01T00:00:00Z"}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make the regression exercise the text-based anchor

The two fixtures still use different roles (user and model), while the pre-change fallback already included role plus timestamp. Making their timestamps equal therefore did not make the old implementation collide: both assertions passed before this commit, and reverting the newly added text component would remain green. Use the same role and timestamp with different text, or add an equivalent case, so this test actually fails under the timestamp-anchor implementation it claims to guard against.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. The Drive owner regression now uses the same role and timestamp for the idless records, and the focused Drive/parser route remains green. This old test-shape concern is covered by the updated real-route evidence.

@Sinity

Sinity commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: c1af5cc256

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/unit/cli/test_query_expression.py`:
- Line 2279: Update the `_mid` calls at the five affected fixture selections to
pass the selected message’s zero-based position explicitly: use positions 1, 1,
2, 2, and 3 in the order identified, while preserving the existing message IDs
and selection expectations.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 983b038d-182a-4c46-aa43-868fbab80197

📥 Commits

Reviewing files that changed from the base of the PR and between e386ccc and c1af5cc.

📒 Files selected for processing (52)
  • .beads/issues.jsonl
  • docs/plans/reindex-incident-coverage.json
  • polylogue/api/archive.py
  • polylogue/daemon/web_shell.py
  • polylogue/operations/mutation_actuators.py
  • polylogue/pipeline/ids.py
  • polylogue/sources/origin_specs.py
  • polylogue/sources/parsers/base_support.py
  • polylogue/storage/raw_authority.py
  • polylogue/storage/sqlite/archive_tiers/archive.py
  • polylogue/storage/sqlite/archive_tiers/user_write.py
  • tests/benchmarks/test_full_session_replace.py
  • tests/benchmarks/test_graph_resolve_deferred_tail.py
  • tests/fixtures/reindex_incident_coverage/campaign_graph.json
  • tests/infra/identity.py
  • tests/property/test_fts_identity_state_machine.py
  • tests/property/test_write_path_state_machine.py
  • tests/unit/api/test_facade_contracts.py
  • tests/unit/archive/query/test_execution_control.py
  • tests/unit/archive/test_with_units_projection.py
  • tests/unit/cli/test_query_exec_laws.py
  • tests/unit/cli/test_query_expression.py
  • tests/unit/daemon/test_convergence_stages.py
  • tests/unit/daemon/test_daemon_http_contracts.py
  • tests/unit/daemon/test_web_reader.py
  • tests/unit/devtools/test_incident_coverage_ledger.py
  • tests/unit/pipeline/test_branching.py
  • tests/unit/sources/test_claude_code_normalization_laws.py
  • tests/unit/sources/test_parsers_drive.py
  • tests/unit/storage/test_archive_search_contracts.py
  • tests/unit/storage/test_archive_tiers_archive.py
  • tests/unit/storage/test_archive_tiers_assertions.py
  • tests/unit/storage/test_archive_tiers_ddl.py
  • tests/unit/storage/test_attachment_acquisition.py
  • tests/unit/storage/test_attachment_reacquisition.py
  • tests/unit/storage/test_fts_identity_ledger.py
  • tests/unit/storage/test_fts_repair_sql.py
  • tests/unit/storage/test_incremental_rebuild_equivalence.py
  • tests/unit/storage/test_lineage_normalization.py
  • tests/unit/storage/test_marks_identity_preserving.py
  • tests/unit/storage/test_message_query_reads.py
  • tests/unit/storage/test_pl_fold.py
  • tests/unit/storage/test_query_unit_time_expression.py
  • tests/unit/storage/test_raw_authority_ledger.py
  • tests/unit/storage/test_schema_safety.py
  • tests/unit/storage/test_search_timeless_since_filter.py
  • tests/unit/storage/test_session_insight_refresh.py
  • tests/unit/storage/test_spec_driven_hydration.py
  • tests/unit/storage/test_store_ops.py
  • tests/unit/storage/test_unread_wire_batch_v46.py
  • tests/unit/storage/test_user_state_target_kinds.py
  • tests/visual/conftest.py

Comment thread tests/unit/cli/test_query_expression.py Outdated
@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex review

Please review exact head 7a777f71acc60af8f265e3cfb954a9b52e03a5ba.

@chatgpt-codex-connector

ghost commented Aug 9, 2026

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 7a777f71ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex review Please perform a fresh substantive review of PR #3898 at exact head 19821c2. Prior Codex comments target older commits. The current CodeRabbit result is rate limited and does not provide substantive review.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19821c2753

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py Outdated
if message.timestamp:
return f"{_TIMESTAMP_ANCHOR_PREFIX}:{message.role}:{message.timestamp}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'text': _normalize_for_hash(message.text)})}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload(_message_comparison_payload(message))}"

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Disambiguate duplicate idless attachment owners

When two idless messages have identical role, timestamp, text, and structured blocks—for example, a duplicated Drive turn—this content-only fallback gives both messages the same owner anchor. Moving an attachment from one duplicate occurrence to the other then leaves both the message payloads and the attachment's anchored payload unchanged, so session_content_hash() matches and re-ingest skips updating the persisted owner. Include a stable duplicate-occurrence discriminator so attachment reassignment between otherwise identical messages changes the session hash.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 4a6dd1f. Duplicate idless owner anchors now receive a private discriminator from normalized message coordinates only in the attachment hash path. Public provider message IDs remain native or empty. The production ArchiveStore regression proves reassignment changes content_hash instead of incrementing skipped_sessions, and the reorder regression proves the owner set remains stable when retained parsed positions are reordered.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Duplicate idless owner keys now use reorder-stable provider evidence when available and raise MessageOwnerAmbiguityError when evidence is indistinguishable. The real Claude reorder test reverses raw records and checks attachment ownership.

@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex review Please perform a fresh substantive review of PR #3898 at exact head 4a6dd1fa60c4394185ed22a860f4753f0e00de93. Recheck duplicate idless owner discrimination, Claude native-versus-private evidence-key separation, public empty IDs, reorder stability, owner reassignment hashing, and the explicit partial disposition with named residual successors.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a6dd1fa60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py Outdated
Comment on lines +302 to +306
occurrence = hash_payload(
{
"position": message.position,
"variant_index": _normalize_for_hash(message.variant_index),
}

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep duplicate-owner anchors stable across reorder

When two idless messages have identical comparison payloads and one owns an attachment, a provider reordering those raw records changes the parser-assigned position, so this discriminator changes even though the attachment stayed with the same source record. That changes attachment_identities and session_hash, turning a harmless re-export reorder into a new revision or raw-authority conflict. Fresh evidence at this head is the new position-derived occurrence discriminator; the added regression only reverses message objects whose position fields remain unchanged, so it does not exercise parser reassignment. Derive the private owner key from reorder-stable evidence rather than the normalized position.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Duplicate idless owners now use reorder-stable provider evidence when available, while indistinguishable duplicates raise typed MessageOwnerAmbiguityError. The regression reverses raw Claude parser input and verifies attachment ownership, so it does not reuse already-positioned messages.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Claude owner evidence includes attachment IDs even when explicit position evidence exists. The new repeated-position, omitted-variant anti-vacuity test fails if attachment evidence is removed from _owner_stable_key.

attachments = _merge_attachment_rows([attachment for evidence in emitted for attachment in evidence.attachments])
attachments = _merge_attachment_rows(
[
attachment.model_copy(update={"message_position": position_by_id[evidence.evidence_key]})

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the full Claude attachment owner coordinate

When two idless Claude branch messages share a position but have different variant_index values, this gives attachments from both variants the same position-only owner coordinate. Both _message_owner_anchors and the production writer's _attachment_message_id_maps key their lookup maps only by that position, so the later variant overwrites the earlier one and an attachment nested under the earlier branch is hashed and stored against the wrong message. Fresh evidence at this head is the newly added position-only association; carry the full (position, variant_index) coordinate through the private attachment linkage.

AGENTS.md reference: AGENTS.md:L55-L60

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Claude attachments now carry the private MessageOwnerCoordinate with both position and variant_index through parser normalization. Hashing, archive writing, and attachment relink use the shared owner-resolution contract; the duplicate-native same-position writer test proves both variants retain their attachments.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Claude attachment ownership carries the full private coordinate, including position and variant index, through normalization and the shared hash/write/relink owner-resolution path. The duplicate-native same-position writer regression proves both variants retain their attachments.

Comment thread polylogue/pipeline/ids.py Outdated
Comment on lines +275 to +277
if message.provider_message_id:
return message.provider_message_id
if message.timestamp:
return f"{_TIMESTAMP_ANCHOR_PREFIX}:{message.role}:{message.timestamp}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'text': _normalize_for_hash(message.text)})}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload(_message_comparison_payload(message))}"

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep timestamped idless identity stable across edits

When a provider re-exports an idless message after editing its text or structured blocks while retaining the original role and creation timestamp, this content-derived value changes even though it is used as the message's identity as well as its content. _message_axis_relation then sees the old and new revisions as mutually disjoint identities rather than one identity with revised content, so a cohort with an existing accepted head is quarantined as ambiguous instead of applying the update. This regression is new for timestamped idless messages—the previous fallback kept role and timestamp as the stable identity anchor—so use a content-independent discriminator for matching same-timestamp occurrences rather than folding mutable message content into the identity itself.

AGENTS.md reference: AGENTS.md:L118-L122

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Timestamped idless revision matching now uses only role and timestamp. Mutable text and blocks remain in the content payload, so the real Claude ingest test proves shared revision identity, changed session hash, and persisted edited block text.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Timestamped idless revision identity is now role plus timestamp only; text and blocks stay in the mutable content payload. The real edit test proves identity continuity and changed persisted content.

Comment thread .beads/issues.jsonl Outdated
{"_type":"issue","id":"polylogue-s8s54","title":"Retroactive origin reclassification for 41 pre-fix browser-capture ChatGPT raws (mvq8 unmet AC2, S-class)","description":"S-class (durable-tier), adjudicated 2026-08-03 (reindex-gate-hunt task #14 Finding 2) as duplicate of polylogue-mvq8's UNMET AC(2) — successor bead carrying exactly that deferred scope.\n\nEvidence: 41 of 844 raw_sessions with source_path under /browser-capture/chatgpt/ carry origin=unknown-export (4.9%); the other 803 are chatgpt-export. Root cause CLOSED by blob-level diff (hunter-parser): NOT a live detector bug — all 41 acquired BEFORE PR #3558 (merged 2026-08-02, mvq8 AC1), which fixed the provider marker sitting past the old 1MiB prefix-scan window (raw_provider_payload sorts alphabetically before session.provider under sort_keys=True). Envelope shape byte-identical between groups; session.provider==\"chatgpt\" present and valid JSON in the misdetected blobs. New acquisitions classify correctly today.\n\nWhy S / why before 818fy: origin is stamped durably at acquisition into source.db and never re-derived; sessions.session_id is a GENERATED column origin||\":\"||native_id. The 818fy reparse would bake the wrong origin AND wrong session identity in again for these rows; fixing after means a second identity churn. Repair machinery already exists: polylogue/storage/unknown_export_reclassification.py (read-only classifier built for exactly this). Scope = pure repair run + receipt, NO parser change.\n\nOne of the 41 is also one of the 6 unmaterialized raws cited in the f1vg audit-trail note (task #14 Finding 1) — reclassification may unblock its materialization; verify while in the area.\n","acceptance_criteria":"The 41 raw_sessions rows under /browser-capture/chatgpt/ with origin=unknown-export are reclassified to chatgpt-export in source.db via an operator-authorized apply pass driven by storage/unknown_export_reclassification.py (or successor actuator). Post-repair query shows 0 unknown-export rows on that acquisition path. Repair receipt recorded. 818fy reparse then derives correct session_id identity for all 41.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:18:21Z","created_by":"Sinity","updated_at":"2026-08-03T12:18:21Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-04T08:00:51Z","started_at":"2026-08-04T08:00:51Z","lease_expires_at":"2026-08-04T08:05:51Z","heartbeat_at":"2026-08-04T08:00:51Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.\n\n2026-08-08 correction for PR #3898: this Bead remains active. The branch satisfies the production positional-identity comparison repair, idless multiplicity and attachment-owner coverage, canonical generated-ID fixture migration, and durable session scope_ref persistence for newly written message marks and annotations. It does not backfill pre-existing message assertions whose scope_ref is absent, so those assertions remain unresolved after index loss. The named successor polylogue-message-owner-scope-backfill owns the typed, backup-gated pre-reindex backfill; no production apply is authorized in PR #3898. The default affected-test verification is recorded under polylogue-93xe because the checkout cannot seed testmon until the separate repair lands.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-08T23:33:21Z","started_at":"2026-08-04T08:00:51Z","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"polylogue-message-owner-scope-backfill","title":"Backfill durable message assertion owners before reindex","description":"Pre-reindex repair for legacy message-target assertions whose durable scope_ref owner is absent. The backfill must use typed assertion and indexed-message records, never delimiter parsing, and must finish before any index reset or semantic reindex that could remove the current owner lookup.","acceptance_criteria":"1. Inventory every active or candidate message-target assertion with a missing durable scope_ref through the typed user-tier route and produce a bounded audit count.\n2. Resolve each owner only from the indexed messages.session_id relation; ambiguous or missing owners produce a report and abort without mutation.\n3. Verify a backup manifest before mutation, update owners transactionally and idempotently as scope_ref=session:<session_id>, and preserve assertion identity and timestamps.\n4. Verify zero eligible legacy rows remain, with a real-route regression proving the owner survives index loss and reindex preparation.\n5. Record the backup, mutation, and post-check receipts before the 818fy pre-reindex boundary. No production apply is authorized from PR #3898.","notes":"Created as the explicit residual from PR #3898. That branch persists scope_ref=session:<id> for new message assertions and intentionally leaves legacy unscoped assertions unresolved after index loss. This successor owns the typed inventory, verified-backup gate, transactional backfill, and pre-reindex receipts. A separate stacked worktree is implementing the successor; this record is the durable scope boundary only.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-08-08T23:33:21Z","created_by":"Sinity","updated_at":"2026-08-08T23:33:21Z","labels":["area:storage","lane:reindex","tracker:internal-only"],"dependencies":[{"issue_id":"polylogue-message-owner-scope-backfill","depends_on_id":"polylogue-slshy","type":"blocks","created_at":"2026-08-08T23:33:21Z","created_by":"Sinity","metadata":"{\"purpose\":\"legacy-message-assertion-owner-backfill\"}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include batch-scoped annotations in the owner backfill

The successor inventories only message assertions with a missing scope_ref, but schema/batch annotation labels already have a nonempty scope_ref shaped as annotation-batch:<id>. Those durable message-target annotations are therefore excluded from this backfill even though _user_state_session_ids cannot obtain their owner from that scope and will lose the indexed fallback during the planned reset; changing their scope to session:<id> would instead destroy their batch provenance. Extend the residual design to preserve the annotation-batch scope while persisting the owner separately, otherwise the explicit partial disposition still leaves batch labels undiscoverable by session after reindex.

AGENTS.md reference: AGENTS.md:L131-L134

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retained as explicit partial scope outside this PR. The remote successor feature/maintenance/message-owner-scope-backfill remains the owner and must preserve annotation-batch:<id> provenance while adding durable session ownership separately. The PR matrix and exact-head carrier keep that residual visible.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retained as explicit partial scope. The branch does not close the bead or perform the durable backfill. The updated successor contract preserves annotation-batch:<id> provenance while adding separate durable session ownership, and the exact-head carrier records that residual.

@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex Please perform a fresh substantive exact-head review of PR #3898 at b4d9580. Recheck the four P1 identity paths end to end: timestamped idless edit continuity with changed persistence, Claude private (position, variant_index) attachment ownership, reorder-stable duplicate idless evidence with typed ambiguity for indistinguishable duplicates, and duplicate-native hash/writer agreement at the raw-ingest skip boundary. Also verify the explicit partial batch-scoped annotation successor residual and do not treat it as satisfied.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4d9580ce6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py Outdated
Comment on lines +333 to +336
if revision_counts[revision_id] == 1:
key = revision_id
elif content_counts[content_id] == 1:
key = content_id

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve timestamp identity when sibling timestamps collide

When two idless messages share a role and timestamp but have different bodies, this branch replaces their role/timestamp revision identity with the mutable content hash. If one message is later edited without changing its timestamp, its identity changes from the old-content hash to the new-content hash; because the unchanged sibling remains present, _message_axis_relation sees mutually unmatched identities and raw-authority processing can quarantine the revision against an existing head instead of persisting the edit. Fresh evidence beyond the earlier single-message repair is this exact-head duplicate branch—the regression covers only a session containing one message. Use content-independent stable evidence for the collision, or fail with typed ambiguity rather than making mutable content the revision identity.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. Same-role same-timestamp idless siblings retain one mutable revision axis, while membership compares multiplicity for that axis rather than treating changed sibling content as a native-ID conflict. The new sibling-edit regression is green.

Comment thread .beads/issues.jsonl Outdated
{"_type":"issue","id":"polylogue-s8s54","title":"Retroactive origin reclassification for 41 pre-fix browser-capture ChatGPT raws (mvq8 unmet AC2, S-class)","description":"S-class (durable-tier), adjudicated 2026-08-03 (reindex-gate-hunt task #14 Finding 2) as duplicate of polylogue-mvq8's UNMET AC(2) — successor bead carrying exactly that deferred scope.\n\nEvidence: 41 of 844 raw_sessions with source_path under /browser-capture/chatgpt/ carry origin=unknown-export (4.9%); the other 803 are chatgpt-export. Root cause CLOSED by blob-level diff (hunter-parser): NOT a live detector bug — all 41 acquired BEFORE PR #3558 (merged 2026-08-02, mvq8 AC1), which fixed the provider marker sitting past the old 1MiB prefix-scan window (raw_provider_payload sorts alphabetically before session.provider under sort_keys=True). Envelope shape byte-identical between groups; session.provider==\"chatgpt\" present and valid JSON in the misdetected blobs. New acquisitions classify correctly today.\n\nWhy S / why before 818fy: origin is stamped durably at acquisition into source.db and never re-derived; sessions.session_id is a GENERATED column origin||\":\"||native_id. The 818fy reparse would bake the wrong origin AND wrong session identity in again for these rows; fixing after means a second identity churn. Repair machinery already exists: polylogue/storage/unknown_export_reclassification.py (read-only classifier built for exactly this). Scope = pure repair run + receipt, NO parser change.\n\nOne of the 41 is also one of the 6 unmaterialized raws cited in the f1vg audit-trail note (task #14 Finding 1) — reclassification may unblock its materialization; verify while in the area.\n","acceptance_criteria":"The 41 raw_sessions rows under /browser-capture/chatgpt/ with origin=unknown-export are reclassified to chatgpt-export in source.db via an operator-authorized apply pass driven by storage/unknown_export_reclassification.py (or successor actuator). Post-repair query shows 0 unknown-export rows on that acquisition path. Repair receipt recorded. 818fy reparse then derives correct session_id identity for all 41.","status":"open","priority":1,"issue_type":"bug","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:18:21Z","created_by":"Sinity","updated_at":"2026-08-03T12:18:21Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-04T08:00:51Z","started_at":"2026-08-04T08:00:51Z","lease_expires_at":"2026-08-04T08:05:51Z","heartbeat_at":"2026-08-04T08:00:51Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"polylogue-slshy","title":"Positional provider_message_id: fix the 18 parser call sites gysk3 deferred (K-class, gates xselt stamps)","description":"K-class (fsgdd taxonomy, adjudicated 2026-08-03 by the reindex-gate-hunt team; task #11). gysk3 closed via PR #3604 fixing only the symptom at the identity-hash call site (_message_comparison_id prefers role+timestamp content anchor when provider_message_id is EMPTY). Root cause deferred with NO successor bead: 18 call sites across 10 parser files bake positional strings directly INTO provider_message_id, so the fixed fallback never fires — the synthetic id arrives indistinguishable from a native one (ids.py's own docstring records this).\n\nCall sites (verbatim from gysk3, 3/18 re-verified live by adjudicator): claude/common.py:912-914 f\"msg-{index}\"; claude/code_parser.py:1620 str(record_uuid or f\"msg-{index}\"); codex.py:1592,1895,1931,2010,2200,2385 (function-call-/reasoning-/compaction-summary-{index}); local_agent.py:205,252; grok.py:139 (unconditional — never has a real id); drive.py:345 chunk-{idx}; chatgpt.py:604; base_support.py:360 (shared builder — inherited by every parser routing through it); antigravity.py:414.\n\nRealism: full-re-export/replay providers (ChatGPT, Claude.ai, Grok, Drive, Antigravity) are high-realism — reordering across acquisitions is already PROVEN for ChatGPT on two sibling axes (c429 message order, uqwd event anchoring). Append-only Codex/Claude Code are low-realism but same pattern.\n\nWhy K: message identity feeds session_revision_projection (acquire-time revision comparison writing durable decisions) AND xselt bootstrap stamps would be computed over a position-derived identity axis — the same failure class the attachment fix (hith/d8al) removed by excluding synthetic ids from identity entirely. fsgdd first-pass sweep already rated the gysk3 family K.\n\nDESIGN (recommended shape b, the attachment_identity_hash precedent): stop filling provider_message_id with positional strings — leave it empty and let _message_comparison_id fallback run. Where a parser needs a per-message unique key for its OWN internal bookkeeping, keep it local, never in ParsedMessage.provider_message_id. Alternative shape a (typed positionally-synthesized marker on ParsedMessage honored by _message_comparison_id) is acceptable if shape b breaks a provider whose ids are genuinely stable-but-synthetic. base_support.py:360 is the highest-leverage single edit.\n\nCAUTION: this changes message identity for affected rows — it MUST land before xselt writes bootstrap stamps and rides the 818fy full reparse (blocks edges express this). Landing it after stamps would poison the identity baseline for every future differential reparse.\n","acceptance_criteria":"No parser fills provider_message_id with an array-position-derived string; id-less messages flow through _message_comparison_id content-anchor fallback (role+timestamp). docs/plans/position-derived-identity-acks.json deleted or emptied. Focused tests: reordered re-export of an id-less message keeps a stable comparison identity per provider fixture. xselt premise note updated to reference this bead as landed.","notes":"Executability pass (iteration 5, 2026-08-03; top-down constraints, executor chooses details): live count is now ~25 positional sites (rg 'provider_message_id\\s*=.*(position|idx|index|f\")' polylogue/sources/parsers/), grown past gysk3's 18 — re-derive the list at execution time, don't trust either count. The sites split into TWO classes with different fixes: (A) NATIVE-ID-WITH-POSITIONAL-FALLBACK ('or f\"...-{index}\"' shapes, e.g. codex.py:1593/2201, local_agent.py:214, code_parser.py:1620): replace the positional fallback with None so _message_comparison_id's content-anchor fallback runs (the gysk3 fix made that fallback trustworthy; attachment_identity_hash is the precedent). (B) PURE-SYNTHETIC rows (summaries/artifacts/reasoning: grok.py:139, antigravity.py:425, codex reasoning-{index}/compaction-summary-{idx}, hermes *-summary): these have no native id by nature — give them a content-derived stable suffix (hash of payload) OR leave positional but mark ParsedMessage positionally_synthesized so comparison ignores the id; pick ONE mechanism repo-wide, don't mix. BOUNDARY: never touch a site where a real native id exists; do not change hermes fixed-string ids ('{session}:system' etc are singleton-stable, not positional — exclude them from the fix set). RED FIRST: zoo 'vintage-reorder' entry (ey4ro row 1) must fail before, pass after. Delete docs/plans/position-derived-identity-acks.json (the suppression registry) in the same PR — its 18 entries reference closed gysk3.\n\n2026-08-03 partial-implementation session: fixed all 9 class-A (native-id-with-positional-fallback) sites -- base_support.py:394, codex.py:1593/1896/1932/2201, local_agent.py:214/261, claude/code_parser.py:1405/1620. Positional f\"msg-{index}\"-style fallbacks replaced with empty string, letting _message_comparison_id's content-anchor (role+timestamp) fallback run per the bead's own shape-b design. Storage layer already correctly maps an empty/whitespace native id to NULL (_stored_message_native_id, pre-existing), so message_id's generated-column COALESCE(native_id, position.variant_index) fallback fires as designed -- no storage-layer change needed.\n\nSurfaced and fixed a real latent bug in the same pass: claude/code_parser.py's EAGER path (_parse_code_records, line ~1783) flagged is_active_leaf by comparing provider_message_id string equality against the last message's id -- the exact naive bug mark_last_occurrence_as_active_leaf (base_support.py) was already built to avoid for the STREAMING path, with its own docstring explaining why (duplicate/empty ids can match more than one position). This was previously masked because unique positional \"msg-N\" strings never collided; once positional fallbacks were removed, two id-less messages sharing \"\" both flagged is_active_leaf=True in the eager path (confirmed via the test_claude_code_normalization_laws.py eager-vs-streamed equality test, which caught it immediately). Not yet fixed -- needs the same by-position fix as mark_last_occurrence_as_active_leaf; left for the next continuation since it's now a known, isolated, well-scoped follow-up (not blocking the class-A landing).\n\nUpdated tests/unit/sources/test_claude_code_normalization_laws.py's two id-less fixture positions (formerly asserting the OLD \"msg-7\"/\"msg-11\" positional-fallback behavior as correct) to assert the new content-anchor/NULL-native-id behavior instead; added _EXPECTED_MAIN_NATIVE_IDS to separate the in-memory (\"\" pre-storage) from post-storage (None, via _stored_message_native_id) expectations, which now legitimately diverge.\n\nVerification: devtools test tests/unit/sources/test_parsers_codex.py tests/unit/sources/test_parsers_local_agent.py tests/unit/sources/test_claude_code_normalization_laws.py tests/unit/sources/test_parsers_props.py tests/unit/sources/test_parsers_claude_code_artifacts.py -> 200 passed, 1 known pre-existing failure (polylogue-4987i, unrelated eager/stream session_events ordering bug, already tracked). mypy --strict clean on all 4 touched files.\n\nRemaining: (1) fix the newly-surfaced is_active_leaf-by-position bug in code_parser.py's eager path before this can be considered fully safe to land; (2) class-B pure-synthetic sites (grok.py, antigravity.py, codex reasoning/compaction-summary) still need the \"pick ONE mechanism repo-wide\" design decision the bead's own notes require -- not attempted this session; (3) delete docs/plans/position-derived-identity-acks.json (the gysk3-referencing suppression registry) once the full fix lands, per the bead's own instruction; (4) has NOT been run against a broader affected-area sweep yet (tests/unit/sources/ + tests/unit/pipeline/ in progress).\n\n2026-08-03 continuation: fixed the is_active_leaf-by-position bug this session's own earlier note flagged as a remaining item (code_parser.py eager path now mirrors mark_last_occurrence_as_active_leaf's by-position approach). Full verification clean: devtools verify --quick green after regenerating docs/examples/demo-tour/ evidence (message-identity change shifted the demo's completion-claim sample manifest hash, expected fallout) and dropping 10 stale entries from docs/plans/position-derived-identity-acks.json (per devtools lab policy position-derived-identity's own fix instruction -- these were exactly the class-A sites this session fixed). Committed as dd5a3445e (the fix) + a4c2c6bbb (evidence/acks regeneration), pushed to master. Class-A scope (9 sites) is now fully landed and verified. Class-B (grok.py, antigravity.py, codex reasoning/compaction-summary) remains open, needing the repo-wide \"pick ONE mechanism\" design decision per this bead's own notes.\n2026-08-04 adversarial review findings: (1) session_revision_projection stores message identity/content in a set, so one versus two identical timestamp-less synthetic records compare equal although the writer persists different multiplicity; preserve unordered multiplicity and add a membership regression. (2) Empty native IDs enter Codex and Drive parent chaining, then writer discards empty parent IDs; carry parser-local parent coordinates resolved by writer without persisting them as native IDs, with parse-to-archive tests. (3) Drive attachment message_position selects the correct writer target but attachment hashing sees only empty message_provider_id; include a stable non-positional owner comparison anchor and test attachment moves between id-less messages change content and revision identity. PR #3730 is therefore insufficient as xselt prerequisite.\n\n2026-08-08 correction for PR #3898: this Bead remains active. The branch satisfies the production positional-identity comparison repair, idless multiplicity and attachment-owner coverage, canonical generated-ID fixture migration, and durable session scope_ref persistence for newly written message marks and annotations. It does not backfill pre-existing message assertions whose scope_ref is absent, so those assertions remain unresolved after index loss. The named successor polylogue-message-owner-scope-backfill owns the typed, backup-gated pre-reindex backfill; no production apply is authorized in PR #3898. The default affected-test verification is recorded under polylogue-93xe because the checkout cannot seed testmon until the separate repair lands.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"Sinity","owner":"ezo.dev@gmail.com","created_at":"2026-08-03T12:17:58Z","created_by":"Sinity","updated_at":"2026-08-08T23:33:21Z","started_at":"2026-08-04T08:00:51Z","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"polylogue-message-owner-scope-backfill","title":"Backfill durable message assertion owners before reindex","description":"Pre-reindex repair for legacy message-target assertions whose durable scope_ref owner is absent. The backfill must use typed assertion and indexed-message records, never delimiter parsing, and must finish before any index reset or semantic reindex that could remove the current owner lookup.","acceptance_criteria":"1. Inventory every active or candidate message-target assertion with a missing durable scope_ref through the typed user-tier route and produce a bounded audit count.\n2. Resolve each owner only from the indexed messages.session_id relation; ambiguous or missing owners produce a report and abort without mutation.\n3. Verify a backup manifest before mutation, update owners transactionally and idempotently as scope_ref=session:<session_id>, and preserve assertion identity and timestamps.\n4. Verify zero eligible legacy rows remain, with a real-route regression proving the owner survives index loss and reindex preparation.\n5. Record the backup, mutation, and post-check receipts before the 818fy pre-reindex boundary. No production apply is authorized from PR #3898.","notes":"Created as the explicit residual from PR #3898. That branch persists scope_ref=session:<id> for new message assertions and intentionally leaves legacy unscoped assertions unresolved after index loss. This successor owns the typed inventory, verified-backup gate, transactional backfill, and pre-reindex receipts. A separate stacked worktree is implementing the successor; this record is the durable scope boundary only.","status":"open","priority":1,"issue_type":"task","owner":"ezo.dev@gmail.com","created_at":"2026-08-08T23:33:21Z","created_by":"Sinity","updated_at":"2026-08-08T23:33:21Z","labels":["area:storage","lane:reindex","tracker:internal-only"],"dependencies":[{"issue_id":"polylogue-message-owner-scope-backfill","depends_on_id":"polylogue-slshy","type":"blocks","created_at":"2026-08-08T23:33:21Z","created_by":"Sinity","metadata":"{\"purpose\":\"legacy-message-assertion-owner-backfill\"}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Record batch-scoped labels in the owner-backfill successor

For message-target annotation labels, scope_ref is already annotation-batch:<id>, so this successor's inventory of only rows with a missing scope_ref excludes them; replacing that scope with session:<id> would also destroy their batch provenance. After the planned index reset, _user_state_session_ids therefore still has neither a durable session owner nor its indexed fallback for those labels. Fresh evidence at this exact head is that the committed successor acceptance criteria still neither inventory non-session scopes nor specify a separate durable owner field, despite the review reply describing that residual as retained. Add the batch-scoped case explicitly to the successor's scope and acceptance contract; it remains partial and must not be treated as satisfied.

AGENTS.md reference: AGENTS.md:L131-L134

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retained as explicit partial scope. The branch updates polylogue-message-owner-scope-backfill to inventory both missing-scope message assertions and annotation-batch:<id> rows, preserve the batch scope, and store session ownership separately. No production backfill is claimed here.

Comment thread polylogue/pipeline/ids.py Outdated
Comment on lines +350 to +353
by_stable_key = {
coordinate.stable_key: key
for coordinate, key in zip(coordinates, keys, strict=True)
if coordinate.stable_key is not None and key not in ambiguous_keys

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fall back to full coordinates when stable evidence collides

When two idless or duplicate-native Claude messages carry the same explicit position but omit variant_index, normalization later assigns distinct variant indices, but _owner_stable_key was already derived from their identical explicit-position evidence. If their bodies differ, their final owner keys are distinct and neither enters ambiguous_keys, so this dict silently overwrites the duplicated stable key; attachment_message_owner_key() consults that map before the full physical coordinate and consequently assigns attachments from both variants to the last message. Fresh evidence beyond the earlier full-coordinate repair is this exact-head stable-key map—the regression always supplies explicit variant indices and cannot exercise the collision. Count stable-evidence occurrences and fall through to the complete (position, variant_index) coordinate when that evidence is non-unique.

AGENTS.md reference: AGENTS.md:L55-L60

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at exact head and found this specific case already handled: content-discriminating duplicate records get distinct content keys before stable evidence is used, while attachment evidence is also included in the stable key. The new repeated-position evidence test covers the formerly colliding case; no code change is needed for the stated bodies-differ case.

Comment thread polylogue/pipeline/ids.py Outdated
Comment on lines +274 to +275
if message.provider_message_id:
return message.provider_message_id
if message.timestamp:
return f"{_TIMESTAMP_ANCHOR_PREFIX}:{message.role}:{message.timestamp}"
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload({'role': str(message.role), 'text': _normalize_for_hash(message.text)})}"
return message.provider_message_id.strip()

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat stripped-empty native IDs as idless

When one export represents a missing native message ID as whitespace and another represents it as "", this condition takes different paths: the whitespace value is truthy and returns the stripped empty string as identity, while the empty value receives the role/timestamp content anchor. Storage normalizes both forms to NULL and therefore persists the same position/variant message identity, but their session hashes and revision identities differ, so an otherwise unchanged re-export can miss the raw-ingest skip boundary or be quarantined against an existing accepted head. Test the stripped value for truthiness and route whitespace-only IDs through the idless anchor, matching _stored_message_native_id.

AGENTS.md reference: AGENTS.md:L55-L60

Useful? React with 👍 / 👎.

ghost Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed at exact head. _message_revision_match_id strips provider_message_id before checking truthiness, so whitespace-only and missing IDs take the same fallback path. The projection-equivalence regression and writer whitespace regression are green.

@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@codex review Please perform a fresh substantive exact-head review of PR #3898 at commit 1eb36fdb171210ce6fc5440e6fef621605f5a58c.

Recheck the repaired identity laws, same-role same-timestamp idless sibling edits, whitespace native IDs, duplicate physical coordinates including duplicate native IDs, Claude attachment-owner stability with repeated positions and omitted variants, Codex synthetic public IDs, and the explicit annotation-batch successor scope. Please report only findings that reproduce against this exact head, with file/line evidence and severity. Do not treat the retained annotation backfill as complete; it is intentionally partial and named in the carrier.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1eb36fdb17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py
Comment on lines +341 to +342
elif content_counts[content_id] == 1:
key = content_id

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prefer stable owner evidence before mutable content

When two timestamped idless messages share a role/timestamp but have distinct bodies, this branch selects each body-derived content_id even when the parser supplied stable owner evidence, such as Claude or Drive attachment IDs. If the attachment-owning sibling is later edited without changing its timestamp or attachment, its owner anchor changes; _attachment_hash_payload folds that anchor into the attachment identity, so _relation sees one attachment removed and another added and classifies the legitimate re-export as a conflict. With an existing accepted raw, raw-authority processing therefore quarantines the edit instead of updating the session; prefer unique stable owner evidence before the mutable content discriminator.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

@Sinity
Sinity force-pushed the feature/fix/positional-message-identity-roundout branch from 1eb36fd to 9cac733 Compare August 9, 2026 15:34

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9cac733cac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +647 to +653
attachment_ids = sorted(
(attachment.provider_attachment_id, attachment.provider_file_id, attachment.provider_drive_id)
for attachment in attachments
if attachment.provider_attachment_id or attachment.provider_file_id or attachment.provider_drive_id
)
if attachment_ids:
evidence["attachment_ids"] = [list(values) for values in attachment_ids]

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep attachment identity out of owner evidence

When an idless Claude export has two same-role, same-timestamp turns and a file is reassigned from one turn to the other, these IDs move with the file and therefore produce the same owner_stable_key in both revisions. message_owner_resolution() prefers that key over the owning message's content, and _attachment_hash_payload() then records the same owner anchor; because Claude files are not part of the message blocks, the entire session content hash remains unchanged. Re-ingest consequently skips the revision and leaves attachment_refs pointing to the old message. Use owner-specific stable evidence independent of the attachment itself, or fail closed when none exists.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

Comment thread polylogue/api/archive.py
Comment on lines +6831 to +6834
# A durable user assertion can outlive the rebuildable
# session row. Keep the caller's canonical token so the
# archive read can use its durable message owner scope.
scope_session_id = session_id

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve session aliases after the index row disappears

When the caller supplies a native ID, provider-form ID, or unique prefix rather than the canonical session ID, normal resolution accepts it while the session exists, but after a delete or index reset this fallback passes the unresolved token through unchanged. Durable marks and annotations store their owner as the canonical session:<origin:native_id>, so both list_marks(session_id=...) and the identical annotation path compare that canonical owner against the alias and return no rows. Preserve enough durable identity to resolve the same accepted aliases without index.db, or perform an ambiguity-checked match against durable canonical owners.

AGENTS.md reference: AGENTS.md:L118-L122

Useful? React with 👍 / 👎.

@Sinity

ghost commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@circleci rerun

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3ea5649ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +121 to +122
if identity in mutable_identities:
continue

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve edited content when comparing idless revisions

When timestamped idless messages retain the same role, timestamp, and multiplicity across an edit, this branch ignores their changed content and classifies the two projections as equal. In the production membership path, if provider_updated_at is absent or unchanged, _equal_content_representative() then chooses the lexicographically smaller content-derived raw_id, so an older export can remain accepted and the edited message is marked superseded_equivalent instead of being materialized. Fresh evidence beyond the previously discussed direct-write regression is this production classifier behavior; represent mutable edits with an ordering that reliably selects the new revision rather than equality with an arbitrary hash tie-break.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

Comment thread polylogue/sources/parsers/drive.py Outdated
Comment on lines +392 to +395
owner_stable_key = (
"drive-owner-evidence:" + hash_payload({"attachments": [list(values) for values in owner_evidence]})
if owner_evidence
else None

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep Drive attachment IDs out of owner evidence

For two same-role, same-timestamp, same-text idless Drive chunks, moving a metadata-only attachment from one chunk to the other leaves this attachment-derived stable key unchanged because it follows the file rather than its owning turn. Such metadata attachments are collected by _collect_chunk_attachments() but need not enter either message's hashed blocks, so the message payloads and attachment payload remain identical and session_content_hash() skips the re-ingest, leaving attachment_refs attached to the old message. Derive the owner key from evidence independent of the attachment, or fail closed when no such evidence exists.

AGENTS.md reference: AGENTS.md:L138-L144

Useful? React with 👍 / 👎.

Sinity added 4 commits August 9, 2026 22:06
Problem:\nId-less provider messages still depended on positional comparison fallbacks, so reordering could manufacture revision changes and same-timestamp Drive messages could collapse attachment ownership. Tagged message IDs also confused reader session ownership.\n\nWhat changed:\n- derive every id-less revision anchor from role, timestamp, and text\n- keep parser-local coordinates separate from provider message identity\n- resolve tagged message IDs back to their owning session\n- cover Drive reorder, parent-chain, and attachment-owner behavior\n\nAlternatives:\n- Retaining timestamp-only or positional fallbacks would lose multiplicity or make attachment moves invisible.\n\nCompatibility:\n- Native provider IDs remain unchanged; synthetic parser IDs continue using the shared content-derived constructor.\n
Problem:\nHand-built archive fixtures still used legacy untagged message and block IDs, causing foreign-key failures and masking the generated identity contract.\n\nWhat changed:\n- add shared test-only constructors backed by the identity law\n- repair storage, query, lineage, API, daemon, property, benchmark, and visual fixtures\n- retain malformed branch-point inputs only where the test exercises repair behavior\n\nAlternatives:\n- Replacing literals mechanically would conceal whether the fixture represents a native or positional message identity.\n\nCompatibility:\n- Production identity generation is unchanged by this test-only sweep; fixtures now match the existing generated-column schema.\n
Problem: parsing canonical message IDs at textual :n: or :p: markers treats opaque provider-native session IDs as delimiters.\n\nWhat changed: list_marks and list_annotations now resolve indexed message ownership through messages.session_id, while retaining the legacy fallback for unresolved or deleted message assertions. Add a real-route regression covering both opaque marker forms.\n\nCompatibility/migration: session targets and non-message targets retain their existing behavior; no schema or production data changes are required.
Problem: the branch closes polylogue-slshy, but the structured 818fy coverage ledger and campaign graph still report it as in progress with no closure path.\n\nWhat changed: mark slshy closed, link the durable polylogue-xselt child successor in both structured records, and update the validator expectation. Receipts remain empty because no live-production proof exists.\n\nCompatibility/migration: the parser-stamps snapshot remains blocking until the named successor or a future live proof satisfies it.
Sinity and others added 12 commits August 9, 2026 22:06
Problem: message marks and annotations lost exact session ownership when rebuildable index rows disappeared, especially for opaque native IDs containing :n: or :p:.

What changed: carry the resolved owner through the async facade, mutation actuators, ArchiveStore, and assertion writers via the existing scope_ref carrier. Reads prefer durable owner metadata, then exact indexed ownership, and bounded legacy rows remain unresolved.

Compatibility/migration: no user schema change. Existing message assertions without durable scope remain unresolved after index loss and require no unsafe delimiter migration.
Problem: changing idless message identity without invalidating revision authority could reuse stale membership verdicts, while per-row owner lookups and the web shell's session-only mark state left related read paths inconsistent.

What changed: advance the revision-membership fingerprint, include structured blocks in idless anchors, batch indexed owner resolution across marks and annotations, filter message marks from session mark state, and strengthen the Drive and authority regressions.

Compatibility/migration: this changes no durable data and performs no production reindex. Existing v2 authority receipts are treated as superseded and are eligible for recomputation under v3.

Review: addressed Codex findings 3742013546, 3742013549, 3742013551, 3742013552, and 3742013555. The older coverage finding 3741792047 remains satisfied by the tracker and incident-ledger correction.
Problem: the exact-head review identified five fixture expectations using the default message position and found that the old per-row owner helper remained beside the batched read path. The structured slshy disposition also needed an explicit regression so its partial state and named residual scope stay aligned with the campaign graph.

What changed: pass canonical positions 1, 1, 2, 2, and 3 in the affected query fixtures, remove the unreferenced single-message owner lookup, exercise one indexed lookup batch for both legacy marks and annotations, and pin slshy's in-progress disposition with both named child beads.

Compatibility/migration: no production data or tracker records changed. Legacy unscoped assertions remain intentionally deferred to polylogue-message-owner-scope-backfill.

Verification: POLYLOGUE_PYTEST_WORKERS=1 devtools test tests/unit/storage/test_marks_identity_preserving.py tests/unit/devtools/test_incident_coverage_ledger.py tests/unit/cli/test_query_expression.py (456 passed, 1 skipped). POLYLOGUE_PYTEST_WORKERS=1 devtools verify --quick (24 steps passed).
Problem: Circle job 2870 read the PR scope carrier before the PR body was refreshed to the exact reviewed head.

What changed: create a signed synchronization commit so the carrier can be re-rendered for the new head before Circle is retriggered.

This commit contains no product-code changes.
Problem: identical idless messages shared one attachment owner anchor, so moving an attachment between duplicate occurrences could leave the session content hash unchanged and make re-ingest skip the ownership change. Claude normalization also needed private evidence keys without publishing synthetic message IDs.

What changed: derive duplicate owner anchors from normalized message coordinates only in the attachment hash path, preserve Claude native IDs separately from private reorder-stable evidence keys, and retain transport positions for Claude attachments. Add parser, revision, Drive-equivalent, and archive re-ingest regressions.

Compatibility: public provider message IDs remain native or empty. The existing partial owner-scope disposition and named successors remain unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
Problem: timestamped idless sibling edits could change revision identity when sibling count changed, while duplicate physical owner coordinates and repeated Claude positions could resolve by last-wins behavior. Whitespace-only native IDs also followed a different revision law than persistence.\n\nWhat changed: keep revision axes separate from private owner keys, compare mutable timestamped idless multiplicity by identity, normalize whitespace native IDs, reject duplicate physical owner coordinates, and retain attachment evidence in repeated-position Claude owner keys. Add real parser, projection, and writer regressions.\n\nCompatibility/migration: public native IDs remain unchanged. No durable tier or index schema changes.
Problem: purely synthetic parser rows exposed content-derived synthetic-* values as public provider message IDs, making mutable text part of public identity.\n\nWhat changed: Codex reasoning and compaction rows, Grok responses, Antigravity markdown rows, and Claude Design synthetic segments now keep provider_message_id empty. Active-leaf selection preserves native IDs and uses physical position for synthetic rows. Claude Design attachments retain private owner coordinates.\n\nCompatibility/migration: native provider IDs and stored public message IDs remain unchanged. Synthetic rows use the existing positional storage fallback and require no schema migration.
Problem: the initial synthetic-id sweep changed established Grok, Antigravity, and Claude Design public contracts beyond the exact branch-local Codex residual.\n\nWhat changed: retain existing synthetic IDs for those established parser contracts and keep the privacy repair limited to Codex reasoning and compaction rows named by the exact review.\n\nCompatibility/migration: no Grok, Antigravity, or Claude Design public identity changes remain. Codex synthetic rows still use empty provider IDs and the canonical storage fallback.
Problem: private attachment ownership could follow mutable message content, and duplicate stable evidence could silently select the last message at one physical coordinate.

What changed: prefer unique stable owner evidence before content discrimination, count stable evidence independently, and require a unique full physical coordinate before resolving duplicated evidence. Add Claude parser, projection, archive write, and re-ingest regressions.

Compatibility: native provider IDs, public idless identity, parser-local lineage, and existing provider contracts remain unchanged.
Problem: idless Claude files could retain an attachment-derived owner key after reassignment, and durable user-state reads could not resolve accepted session aliases once the index row disappeared.\n\nWhat changed: remove attachment identifiers from Claude owner evidence and fail closed when independent owner evidence is absent. Resolve missing-index aliases against canonical active mark and annotation owners with ambiguity checks. Add parser-to-writer, alias, ambiguity, and red-twin regressions.\n\nCompatibility/migration: no schema or production archive migration is performed.\n\nCo-Authored-By: Claude <noreply@anthropic.com>
@Sinity
Sinity force-pushed the feature/fix/positional-message-identity-roundout branch from d3ea564 to e654d98 Compare August 9, 2026 20:09

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e654d98994

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread polylogue/pipeline/ids.py
payload["text"] = _normalize_for_hash(message.text)
if message.blocks and not _is_redundant_text_only_block(message):
payload["content_blocks"] = [_content_block_payload(b) for b in message.blocks]
return f"{_CONTENT_ANCHOR_PREFIX}:{hash_payload(payload)}"

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Namespace fallback identities outside native IDs

When an opaque provider-native ID equals the generated __polylogue_msg_content_anchor__:<hash> for an idless message in the same session, both messages receive the same comparison identity. Because that identity is then marked mutable, _message_axis_relation() ignores content differences for both messages; an edit to the native-ID message can therefore be classified as equal and an older raw retained. Encode native and fallback identities in disjoint hash namespaces rather than relying on a string prefix that native IDs are not prohibited from using.

AGENTS.md reference: AGENTS.md:L55-L60

Useful? React with 👍 / 👎.

event_type="model_configuration",
timestamp=evidence.updated_at or evidence.timestamp,
source_message_provider_id=evidence.provider_message_id,
source_message_provider_id=evidence.native_provider_message_id,

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve event ownership for duplicate native IDs

When Claude emits duplicate native message IDs and either occurrence carries thinking configuration, edit metadata, or web-tool evidence, these events retain only the duplicated native ID. The writer deliberately excludes duplicate IDs from its by_native_id map (write.py:4351-4363), so every such event is persisted with source_message_id=NULL even though normalization now has an exact evidence_key and position/variant coordinate for its originating occurrence. Carry that private coordinate through ParsedSessionEvent so these per-message events remain linked after duplicate occurrences are materialized.

Useful? React with 👍 / 👎.

@Sinity
Sinity merged commit 8c558d3 into master Aug 9, 2026
@Sinity
Sinity deleted the feature/fix/positional-message-identity-roundout branch August 9, 2026 20:26

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 609ff8085b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 872 to 875
explicit_leaf_ids = [
evidence.local_id for evidence in evidence_by_id.values() if evidence.explicit_is_active_leaf is True
evidence.evidence_key for evidence in evidence_by_id.values() if evidence.explicit_is_active_leaf is True
]
if leaf_id is None and len(explicit_leaf_ids) == 1:

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor explicit leaf evidence for duplicate message IDs

When Claude supplies the same native ID for multiple records and the later occurrence is explicitly marked is_active_leaf, a top-level active_leaf_message_provider_id with that duplicated ID already resolves to the first occurrence's unsuffixed evidence key, so this explicit-leaf fallback never runs. The code then overrides leaf_values to mark the first occurrence as the leaf and the actual leaf as false, producing an incorrect active transcript path. Prefer the unique occurrence-level leaf marker whenever the top-level native ID is duplicated, or only resolve the top-level ID when it is unique.

Useful? React with 👍 / 👎.

Comment on lines +996 to +998
occurrence = evidence_key_counts.get(base_evidence_key, 0)
evidence_key_counts[base_evidence_key] = occurrence + 1
evidence_key = base_evidence_key if occurrence == 0 else f"{base_evidence_key}:occurrence:{occurrence}"

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stabilize duplicate evidence keys across record reordering

When duplicate native IDs also share a timestamp and lack explicit positions, these occurrence suffixes are assigned from raw array order and are subsequently used as the final tie-breaker in ordered_flat. Reversing two otherwise unchanged records therefore swaps which body receives the unsuffixed key and reverses their normalized positions; because duplicate native IDs are stored using position/variant identity, a harmless export reorder changes public message IDs and transcript order. Derive the ordering key from reorder-stable record evidence, using occurrence numbering only after that deterministic ordering.

AGENTS.md reference: AGENTS.md:L55-L60

Useful? React with 👍 / 👎.

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@polylogue/api/archive.py`:
- Around line 314-326: Bound the durable owner lookup around the visible
assertions query instead of fetching every active mark and annotation row.
Update the query and supporting logic used by _resolve_user_state_target to
restrict results to plausible session-prefixed target_ref or scope_ref values,
preferably leveraging an index on those columns, while preserving the existing
unresolved-token fallback behavior.
- Around line 344-353: Update _durable_session_alias_matches and its caller
_resolve_durable_user_state_session_id to mirror
ArchiveStore.resolve_session_id’s two-stage resolution: collect and return exact
native-id matches first, and only evaluate native-id prefix matches when the
exact tier is empty. Ensure exact token matches are ranked ahead of prefix
matches so “abc” resolves to origin:abc rather than becoming ambiguous with
origin:abcd, while preserving ambiguity errors within the selected tier.

In `@polylogue/pipeline/ids.py`:
- Around line 373-379: Update the `provider_counts` comprehension in the
surrounding ID-mapping logic to strip `message.provider_message_id` before
testing truthiness, excluding whitespace-only IDs from the counter. Keep the
existing `provider_keys` filtering aligned so empty stripped IDs do not create
false ambiguity for attachments.
- Around line 394-438: Add explicit handling around the session
content-hash/projection and attachment-write boundary that catches
MessageOwnerAmbiguityError from attachment_message_owner_key(). Reject the
session and return before any conn.commit() can persist session or project rows,
ensuring both session_content_hash() and _write_attachments() ambiguity paths
fail closed consistently.

In `@polylogue/sources/parsers/claude/common.py`:
- Around line 1287-1291: Update the active_leaf_message_provider_id assignment
in ClaudeMessageNormalization to convert an empty native_provider_message_id to
None, while preserving None when normalized_active_leaf is absent and retaining
non-empty IDs unchanged.

In `@polylogue/storage/attachment_relink.py`:
- Around line 359-362: Handle MessageOwnerAmbiguityError around
attachment_message_owner_key in the attachment loop within
_match_session_payload, marking only that attachment as
UnrecoverableAttachmentReason.NO_AUTHORITATIVE_RAW (or a dedicated ambiguity
reason) and continuing with the remaining attachments. Import the exception
alongside the existing helpers, while preserving normal owner resolution and
recoverable matching behavior.

In `@polylogue/storage/sqlite/archive_tiers/archive.py`:
- Around line 5511-5528: Choose one consistent contract for unresolved owners
and apply it to both list_marks and list_annotations: either omit ownerless rows
from unfiltered results or retain session_id as an empty string. Update both
methods accordingly and document the behavior in each method’s docstring,
including the existing session-filtered exclusion and the unfiltered case.
Affected site polylogue/storage/sqlite/archive_tiers/archive.py#L5511-L5528
(list_marks) requires the contract and documentation;
polylogue/storage/sqlite/archive_tiers/archive.py#L5694-L5705 (list_annotations)
must apply the same behavior and extend its docstring.

In `@polylogue/storage/sqlite/archive_tiers/user_write.py`:
- Around line 621-638: In polylogue/storage/sqlite/archive_tiers/user_write.py
lines 621-638, update upsert_mark to preserve an existing message owner scope
when owner_session_id is absent; add and reuse a shared helper that returns the
explicit session scope or reads the existing assertion envelope, and use it for
scope_ref. Apply the same helper and preservation rule in upsert_annotation at
lines 752-752; non-message targets should continue producing no owner scope.

In `@tests/unit/archive/test_session_revision_membership.py`:
- Around line 1213-1222: Update the test case for classify_membership_revisions
to use raw IDs whose alphabetical order favors the older revision, while keeping
the newer revision’s observed_at_ms value later. Preserve the expected
accepted_raw_ids result so the assertion can only pass when
_equal_content_representative applies the observed_at_ms tiebreak.

In `@tests/unit/devtools/test_incident_coverage_ledger.py`:
- Around line 62-70: Update the assertions in the incident coverage test to
match the current ledger: expect polylogue-slshy’s bead_status to be closed,
expect residual_successor to reference only polylogue-xselt, and assert the
child_bead_ids set contains only polylogue-xselt.

In `@tests/unit/sources/test_parsers_codex.py`:
- Line 681: Rename test_reasoning_reordering_keeps_synthetic_revision_identity
to reflect that it verifies content-derived revision identity across message
reordering, removing the obsolete synthetic-identity wording while leaving the
test body unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 18c57785-3004-46a1-8ea0-df952388b1ec

📥 Commits

Reviewing files that changed from the base of the PR and between c1af5cc and 609ff80.

📒 Files selected for processing (27)
  • .beads/issues.jsonl
  • docs/plans/reindex-incident-coverage.json
  • polylogue/api/archive.py
  • polylogue/archive/session_revision_membership.py
  • polylogue/core/message_owner.py
  • polylogue/pipeline/ids.py
  • polylogue/sources/parsers/base_models.py
  • polylogue/sources/parsers/base_support.py
  • polylogue/sources/parsers/claude/code_parser.py
  • polylogue/sources/parsers/claude/common.py
  • polylogue/sources/parsers/codex.py
  • polylogue/sources/parsers/drive.py
  • polylogue/sources/parsers/local_agent.py
  • polylogue/storage/attachment_relink.py
  • polylogue/storage/raw_authority.py
  • polylogue/storage/sqlite/archive_tiers/archive.py
  • polylogue/storage/sqlite/archive_tiers/user_write.py
  • polylogue/storage/sqlite/archive_tiers/write.py
  • tests/fixtures/reindex_incident_coverage/campaign_graph.json
  • tests/unit/archive/test_session_revision_membership.py
  • tests/unit/cli/test_query_expression.py
  • tests/unit/devtools/test_incident_coverage_ledger.py
  • tests/unit/pipeline/test_message_identity_position_fallback.py
  • tests/unit/sources/test_claude_web_normalization.py
  • tests/unit/sources/test_parsers_codex.py
  • tests/unit/storage/test_archive_tiers_archive.py
  • tests/unit/storage/test_marks_identity_preserving.py
💤 Files with no reviewable changes (1)
  • .beads/issues.jsonl

Comment thread polylogue/api/archive.py
Comment on lines +314 to +326
try:
with closing(open_readonly_connection(user_db)) as conn:
rows = conn.execute(
"""
SELECT target_ref, scope_ref
FROM assertions
WHERE kind IN (?, ?)
AND COALESCE(status, 'active') != 'deleted'
""",
(AssertionKind.MARK.value, AssertionKind.ANNOTATION.value),
).fetchall()
except sqlite3.Error:
return None

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Performance & Scalability | 🔵 Trivial

Consider bounding the durable owner scan.

This read selects every active mark and annotation row and materializes it with fetchall(), then scans the full canonical-id set in Python for each unresolved token. The fallback fires per unresolved session token, including on write paths through _resolve_user_state_target.

The cost is proportional to total mark and annotation count, not to the number of plausible owners. Two options bound it. Push the alias shapes into SQL so SQLite can filter with an index on scope_ref/target_ref. Or add an index on (kind, status) and select only the distinct session:-prefixed values.

This is advice, not a blocker; the fallback only runs after the index misses.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/api/archive.py` around lines 314 - 326, Bound the durable owner
lookup around the visible assertions query instead of fetching every active mark
and annotation row. Update the query and supporting logic used by
_resolve_user_state_target to restrict results to plausible session-prefixed
target_ref or scope_ref values, preferably leveraging an index on those columns,
while preserving the existing unresolved-token fallback behavior.

Comment thread polylogue/api/archive.py
Comment on lines +344 to +353
def _durable_session_alias_matches(token: str, canonical_id: str) -> bool:
"""Apply the same exact/provider/prefix/suffix alias shapes as the index."""
if token == canonical_id or canonical_id.startswith(token):
return True
if ":" in token:
provider_token, native_id = token.split(":", 1)
provider_origin = origin_from_provider(Provider.from_string(provider_token)).value
return canonical_id == f"{provider_origin}:{native_id}"
_, separator, native_id = canonical_id.partition(":")
return bool(separator and (native_id == token or native_id.startswith(token)))

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Prefer exact matches before prefix matches, as the index resolver does.

The docstring claims the same alias shapes as the index. The suffix branch does not match the index. ArchiveStore.resolve_session_id tries the exact native id first and only widens to a prefix match when the exact lookup finds nothing. That two-stage rule exists so sibling native ids abc and abcd do not make the token abc ambiguous.

Here both shapes are evaluated in one predicate. With durable owners origin:abc and origin:abcd, the token abc matches both, and _resolve_durable_user_state_session_id raises ValueError("session id alias 'abc' is ambiguous"). The index resolves the same token to origin:abc.

The raised ValueError is not caught by the callers. list_marks (Line 6978) and list_annotations (Line 7074) catch only SessionNotFoundError, so the error reaches the API caller. add_mark and save_annotation fail the same way through _resolve_user_state_target.

Rank matches and return the exact tier when it is non-empty.

🐛 Proposed fix to mirror the index resolver's exact-before-prefix rule
-def _durable_session_alias_matches(token: str, canonical_id: str) -> bool:
-    """Apply the same exact/provider/prefix/suffix alias shapes as the index."""
-    if token == canonical_id or canonical_id.startswith(token):
-        return True
-    if ":" in token:
-        provider_token, native_id = token.split(":", 1)
-        provider_origin = origin_from_provider(Provider.from_string(provider_token)).value
-        return canonical_id == f"{provider_origin}:{native_id}"
-    _, separator, native_id = canonical_id.partition(":")
-    return bool(separator and (native_id == token or native_id.startswith(token)))
+def _durable_session_alias_match_rank(token: str, canonical_id: str) -> int | None:
+    """Rank one alias match: 0 is exact evidence, 1 is prefix-widened evidence.
+
+    Mirrors ``ArchiveStore.resolve_session_id``: an exact canonical id, an
+    exact provider-qualified id, or an exact native id outranks any
+    prefix-widened match, so a sibling native id cannot make an exact token
+    ambiguous.
+    """
+    if token == canonical_id:
+        return 0
+    if ":" in token:
+        provider_token, native_id = token.split(":", 1)
+        provider_origin = origin_from_provider(Provider.from_string(provider_token)).value
+        if canonical_id == f"{provider_origin}:{native_id}":
+            return 0
+        return 1 if canonical_id.startswith(token) else None
+    _, separator, canonical_native_id = canonical_id.partition(":")
+    if separator and canonical_native_id == token:
+        return 0
+    if canonical_id.startswith(token):
+        return 1
+    if separator and canonical_native_id.startswith(token):
+        return 1
+    return None

Then select the best-ranked tier in the caller:

-    matches = {canonical_id for canonical_id in canonical_ids if _durable_session_alias_matches(token, canonical_id)}
-    if len(matches) > 1:
-        raise ValueError(f"session id alias {token!r} is ambiguous")
-    return next(iter(matches), None)
+    ranked: dict[int, set[str]] = {}
+    for canonical_id in canonical_ids:
+        rank = _durable_session_alias_match_rank(token, canonical_id)
+        if rank is not None:
+            ranked.setdefault(rank, set()).add(canonical_id)
+    if not ranked:
+        return None
+    matches = ranked[min(ranked)]
+    if len(matches) > 1:
+        raise ValueError(f"session id alias {token!r} is ambiguous")
+    return next(iter(matches))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def _durable_session_alias_matches(token: str, canonical_id: str) -> bool:
"""Apply the same exact/provider/prefix/suffix alias shapes as the index."""
if token == canonical_id or canonical_id.startswith(token):
return True
if ":" in token:
provider_token, native_id = token.split(":", 1)
provider_origin = origin_from_provider(Provider.from_string(provider_token)).value
return canonical_id == f"{provider_origin}:{native_id}"
_, separator, native_id = canonical_id.partition(":")
return bool(separator and (native_id == token or native_id.startswith(token)))
def _durable_session_alias_match_rank(token: str, canonical_id: str) -> int | None:
"""Rank one alias match: 0 is exact evidence, 1 is prefix-widened evidence.
Mirrors ``ArchiveStore.resolve_session_id``: an exact canonical id, an
exact provider-qualified id, or an exact native id outranks any
prefix-widened match, so a sibling native id cannot make an exact token
ambiguous.
"""
if token == canonical_id:
return 0
if ":" in token:
provider_token, native_id = token.split(":", 1)
provider_origin = origin_from_provider(Provider.from_string(provider_token)).value
if canonical_id == f"{provider_origin}:{native_id}":
return 0
return 1 if canonical_id.startswith(token) else None
_, separator, canonical_native_id = canonical_id.partition(":")
if separator and canonical_native_id == token:
return 0
if canonical_id.startswith(token):
return 1
if separator and canonical_native_id.startswith(token):
return 1
return None
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/api/archive.py` around lines 344 - 353, Update
_durable_session_alias_matches and its caller
_resolve_durable_user_state_session_id to mirror
ArchiveStore.resolve_session_id’s two-stage resolution: collect and return exact
native-id matches first, and only evaluate native-id prefix matches when the
exact tier is empty. Ensure exact token matches are ranked ahead of prefix
matches so “abc” resolves to origin:abc rather than becoming ambiguous with
origin:abcd, while preserving ambiguity errors within the selected tier.

Comment thread polylogue/pipeline/ids.py
Comment on lines +373 to +379
provider_counts = Counter(
message.provider_message_id.strip() for message in messages if message.provider_message_id
)
for message, key in zip(messages, keys, strict=True):
provider_id = message.provider_message_id.strip()
if provider_id and provider_counts[provider_id] == 1:
provider_keys[provider_id] = key

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Filter provider_counts on the stripped id, not the raw id.

Line 374 tests message.provider_message_id for truthiness but counts message.provider_message_id.strip(). A whitespace-only native id is truthy and contributes an "" entry to provider_counts. Two such messages put "" into ambiguous_provider_ids. Line 376-379 then skips those messages, so unique_provider_keys never holds "".

The result is asymmetric. An attachment whose message_provider_id is whitespace-only strips to "" at line 432, matches "" in ambiguous_provider_ids, and raises MessageOwnerAmbiguityError at line 434 even though no real provider id is duplicated. Strip before the truthiness test so both sides agree.

🐛 Proposed fix
     provider_counts = Counter(
-        message.provider_message_id.strip() for message in messages if message.provider_message_id
+        message.provider_message_id.strip() for message in messages if message.provider_message_id.strip()
     )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/pipeline/ids.py` around lines 373 - 379, Update the
`provider_counts` comprehension in the surrounding ID-mapping logic to strip
`message.provider_message_id` before testing truthiness, excluding
whitespace-only IDs from the counter. Keep the existing `provider_keys`
filtering aligned so empty stripped IDs do not create false ambiguity for
attachments.

Comment thread polylogue/pipeline/ids.py
Comment on lines +394 to +438
def _attachment_owner_coordinate(attachment: ParsedAttachment) -> MessageOwnerCoordinate:
coordinate = attachment.owner_coordinate
if coordinate is not None:
return MessageOwnerCoordinate(
stable_key=coordinate.stable_key,
position=coordinate.position if coordinate.position is not None else attachment.message_position,
variant_index=coordinate.variant_index,
)
return MessageOwnerCoordinate(
position=attachment.message_position,
variant_index=attachment.message_variant_index or 0,
)


def attachment_message_owner_key(attachment: ParsedAttachment, resolution: MessageOwnerResolution) -> str | None:
"""Resolve one attachment to the same private owner key used by writes."""
coordinate = _attachment_owner_coordinate(attachment)
if (
coordinate.stable_key is not None
and coordinate.stable_key not in resolution.ambiguous_stable_keys
and coordinate.stable_key not in resolution.ambiguous_keys
and coordinate.stable_key in resolution.by_stable_key
):
return resolution.by_stable_key[coordinate.stable_key]
if coordinate.physical_key is not None:
if coordinate.physical_key in resolution.ambiguous_physical_coordinates:
raise MessageOwnerAmbiguityError(f"attachment owner coordinate is duplicated: {coordinate.physical_key!r}")
key = resolution.by_physical_coordinate.get(coordinate.physical_key)
if key is not None:
if key in resolution.ambiguous_keys:
raise MessageOwnerAmbiguityError(
"attachment owner coordinate is indistinguishable from another message: "
f"{coordinate.physical_key!r}"
)
return key
if coordinate.stable_key in resolution.ambiguous_stable_keys:
raise MessageOwnerAmbiguityError(f"attachment owner evidence is duplicated: {coordinate.stable_key!r}")
if attachment.message_provider_id:
provider_id = attachment.message_provider_id.strip()
if provider_id in resolution.ambiguous_provider_ids:
raise MessageOwnerAmbiguityError(
f"attachment provider message id is duplicated without a private coordinate: {provider_id!r}"
)
return resolution.unique_provider_keys.get(provider_id, provider_id)
return None

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Trace MessageOwnerAmbiguityError handling across hash and write paths.
set -euo pipefail

echo "== raise sites =="
rg -nP -C3 '\bMessageOwnerAmbiguityError\b' --type=py

echo "== callers of attachment_message_owner_key =="
rg -nP -C6 '\battachment_message_owner_key\s*\(' --type=py

echo "== callers of session_revision_projection / session_content_hash =="
rg -nP -C4 '\b(session_revision_projection|session_content_hash)\s*\(' --type=py -g '!tests/**'

Repository: Sinity/polylogue

Length of output: 172


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== repo files matching ids.py / archive_tiers / hash functions =="
fd -a 'ids\.py$|write\.py$' polylogue 2>/dev/null | sed 's#^\./##' || true
echo

echo "== all MessageOwnerAmbiguityError occurrences in tracked Python files =="
git ls-files '*.py' | xargs rg -n -C3 '\bMessageOwnerAmbiguityError\b' || true
echo

echo "== all attachment_message_owner_key occurrences in tracked Python files =="
git ls-files '*.py' | xargs rg -n -C5 '\battachment_message_owner_key\s*\(' || true
echo

echo "== all session_revision_projection / session_content_hash occurrences in tracked Python files (excluding tests) =="
git ls-files '*.py' | grep -vE '(^|/)tests?/' | xargs rg -n -C4 '\b(session_revision_projection|session_content_hash)\s*\(' || true
echo

echo "== target file outline =="
ast-grep outline polylogue/pipeline/ids.py --view compact 2>/dev/null || true

Repository: Sinity/polylogue

Length of output: 38111


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== ids.py relevant implementation =="
sed -n '620,775p' polylogue/pipeline/ids.py
echo

echo "== ingest_worker exception handling around content hash/load and write =="
sed -n '700,860p' polylogue/pipeline/services/ingest_worker.py
echo

echo "== revision_governance exception handling for content hash/projection/write =="
sed -n '240,320p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
sed -n '1760,1810p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
sed -n '2148,2175p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
echo

echo "== write.py attachment mapping/write flow =="
sed -n '240,290p' polylogue/storage/sqlite/archive_tiers/write.py
sed -n '3360,3435p' polylogue/storage/sqlite/archive_tiers/write.py
sed -n '3435,3535p' polylogue/storage/sqlite/archive_tiers/write.py
sed -n '3535,3605p' polylogue/storage/sqlite/archive_tiers/write.py
echo

echo "== repo-level MessageOwnerAmbiguityError catchers =="
git ls-files '*.py' | xargs rg -n -C4 'except\s+(.*MessageOwnerAmbiguityError|Exception)\b|raise\s+(.*MessageOwnerAmbiguityError|Exception)\b|MessageOwnerAmbiguityError' || true

Repository: Sinity/polylogue

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== ids.py _session_hash_components and functions =="
sed -n '620,775p' polylogue/pipeline/ids.py
echo

echo "== ingest_worker load/write boundary =="
sed -n '730,795p' polylogue/pipeline/services/ingest_worker.py
echo

echo "== revision_governance boundaries =="
sed -n '260,290p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
sed -n '1776,1810p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
sed -n '2152,2175p' polylogue/storage/sqlite/archive_tiers/revision_governance.py
echo

echo "== revision_governance try blocks around these functions =="
rg -n -C2 'try:|except Exception|except .*:' polylogue/storage/sqlite/archive_tiers/revision_governance.py | grep -n -C2 'write_parsed_session_to_\(load=\|write_parsed_session_to_\(content_hash=\|session_revision_projection\|session_content_hash\|_write_attachments\|MessageOwnerAmbiguityError' || true
echo

echo "== write.py write_parsed_session_to_archive exception handling and attachment block =="
sed -n '230,300p' polylogue/storage/sqlite/archive_tiers/write.py
sed -n '3360,3580p' polylogue/storage/sqlite/archive_tiers/write.py
sed -n '3600,3680p' polylogue/storage/sqlite/archive_tiers/write.py
echo

echo "== write.py try blocks near write_parsed_session_to_archive =="
rg -n -C2 'try:|except Exception|MessageOwnerAmbiguityError' polylogue/storage/sqlite/archive_tiers/write.py | grep -n -C2 'write_parsed_session_to_archive|write_parsed|attachments|owner_key|raw_id|conn\.commit|content_hash' || true
echo

echo "== explicit MessageOwnerAmbiguityError catchers =="
git ls-files '*.py' | xargs rg -n -C3 'except (.*MessageOwnerAmbiguityError|ValueError)\b|except Exception as exc[:\s]*.*MessageOwnerAmbiguityError|MessageOwnerAmbiguityError:' || true

Repository: Sinity/polylogue

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== exact write_parsed_session_to_archive definition/flow =="
rg -n -C3 'def write_parsed_session_to_archive|conn\.commit\(\)|conn\.rollback\(\)|write_parsed_session_to_archive\(|raw_id|prepared=' polylogue/storage/sqlite/archive_tiers/write.py
echo

echo "== revision_governance write function and surrounding connection handling =="
rg -n -C5 'def write_.*session|write_parsed_session_to_archive|session_content_hash|message revision membership|content_unchanged|conn\.commit|conn\.rollback' polylogue/storage/sqlite/archive_tiers/revision_governance.py
echo

echo "== write_parsed_session_to_archive imports and top 280 =="
sed -n '1,280p' polylogue/storage/sqlite/archive_tiers/write.py
echo

echo "== specific ingest_worker full flow around SessionWritePayload =="
sed -n '745,770p' polylogue/pipeline/services/ingest_worker.py
echo

echo "== behavioral probe of shared owner key resolution and error source =="
python3 - <<'PY'
from pathlib import Path
import ast
src = Path("polylogue/pipeline/ids.py").read_text()
tree = ast.parse(src)
for node in ast.walk(tree):
    if isinstance(node, ast.FunctionDef) and node.name == "attachment_message_owner_key":
        print(node.name, node.lineno)
        for child in node.body:
            if isinstance(child, ast.Raise):
                print("  raise line", child.lineno, "exc_type=", ast.get_source_segment(src, child.exc))
    if isinstance(node, ast.FunctionDef) and node.name in {"session_content_hash", "session_revision_projection", "_session_hash_components"}:
        calls = [ast.unparse(n.value.func) for n in ast.walk(node) if isinstance(n, ast.Call) and isinstance(n.value, ast.Name) and n.value.id == "attachment_message_owner_key"]
        print(node.name, node.lineno, "calls attachment_message_owner_key:", bool(calls))
PY

Repository: Sinity/polylogue

Length of output: 29385


Make ambiguous attachment owner resolution fail closed before commit.

session_content_hash() and _write_attachments() both call attachment_message_owner_key(), so both can raise MessageOwnerAmbiguityError for the same input. Add an explicit handler around the hash/projection and write boundary that rejects the session before any conn.commit() can persist the session/project rows for an ambiguous attachment.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/pipeline/ids.py` around lines 394 - 438, Add explicit handling
around the session content-hash/projection and attachment-write boundary that
catches MessageOwnerAmbiguityError from attachment_message_owner_key(). Reject
the session and return before any conn.commit() can persist session or project
rows, ensuring both session_content_hash() and _write_attachments() ambiguity
paths fail closed consistently.

Comment on lines 1287 to 1291
active_leaf_message_provider_id=(
evidence_by_id[normalized_active_leaf].provider_message_id if normalized_active_leaf is not None else None
evidence_by_id[normalized_active_leaf].native_provider_message_id
if normalized_active_leaf is not None
else None
),

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Return None instead of an empty string when the active leaf has no native id.

native_provider_message_id is "" for an id-less message (line 966). When normalized_active_leaf points at such a message, ClaudeMessageNormalization.active_leaf_message_provider_id becomes "" rather than None. The field type is str | None, and the Codex parser now reports None for the same situation (tests/unit/sources/test_parsers_codex.py line 708).

Normalize the empty value so both parsers report absence the same way.

🐛 Proposed fix
         active_leaf_message_provider_id=(
-            evidence_by_id[normalized_active_leaf].native_provider_message_id
+            (evidence_by_id[normalized_active_leaf].native_provider_message_id or None)
             if normalized_active_leaf is not None
             else None
         ),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
active_leaf_message_provider_id=(
evidence_by_id[normalized_active_leaf].provider_message_id if normalized_active_leaf is not None else None
evidence_by_id[normalized_active_leaf].native_provider_message_id
if normalized_active_leaf is not None
else None
),
active_leaf_message_provider_id=(
(evidence_by_id[normalized_active_leaf].native_provider_message_id or None)
if normalized_active_leaf is not None
else None
),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/sources/parsers/claude/common.py` around lines 1287 - 1291, Update
the active_leaf_message_provider_id assignment in ClaudeMessageNormalization to
convert an empty native_provider_message_id to None, while preserving None when
normalized_active_leaf is absent and retaining non-empty IDs unchanged.

Comment on lines +5511 to 5528
selected.append((assertion, found_target_type, found_target_id))
owners = _user_state_session_ids((item[0] for item in selected), index_conn=self._conn)
out: list[dict[str, str]] = []
for assertion, found_target_type, found_target_id in selected:
owner_session_id = owners[assertion.assertion_id]
if session_id and target_id is None and owner_session_id != session_id:
continue
out.append(
{
"target_type": found_target_type,
"target_id": found_target_id,
"session_id": _user_mark_session_id(found_target_type, found_target_id),
"session_id": owner_session_id,
"message_id": found_target_id if found_target_type == "message" else "",
"mark_type": str(assertion.key or ""),
"created_at": str(assertion.created_at_ms),
}
)
return out

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

An unresolvable owner is reported as an empty string. _user_state_session_ids returns "" for a message assertion with neither a durable scope_ref nor an indexed message row, and for any unsupported target type. Both list methods copy that value straight into the result payload's session_id field. A session-filtered call excludes those rows, so the empty owner only escapes on unfiltered calls, where consumers receive {"session_id": ""} and cannot distinguish "owner unknown" from a real id. list_annotations documents the filtered exclusion; list_marks documents nothing.

  • polylogue/storage/sqlite/archive_tiers/archive.py#L5511-L5528: in list_marks, decide the contract for an unresolved owner and document it. Either omit the row from unfiltered results as well, or keep the empty value and state in the docstring that session_id is empty when no owner authority exists.
  • polylogue/storage/sqlite/archive_tiers/archive.py#L5694-L5705: apply the same decision in list_annotations so both readers agree, and extend the existing docstring to cover the unfiltered case.
📍 Affects 1 file
  • polylogue/storage/sqlite/archive_tiers/archive.py#L5511-L5528 (this comment)
  • polylogue/storage/sqlite/archive_tiers/archive.py#L5694-L5705
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/storage/sqlite/archive_tiers/archive.py` around lines 5511 - 5528,
Choose one consistent contract for unresolved owners and apply it to both
list_marks and list_annotations: either omit ownerless rows from unfiltered
results or retain session_id as an empty string. Update both methods accordingly
and document the behavior in each method’s docstring, including the existing
session-filtered exclusion and the unfiltered case. Affected site
polylogue/storage/sqlite/archive_tiers/archive.py#L5511-L5528 (list_marks)
requires the contract and documentation;
polylogue/storage/sqlite/archive_tiers/archive.py#L5694-L5705 (list_annotations)
must apply the same behavior and extend its docstring.

Comment on lines +621 to +638
owner_session_id: str | None = None,
label: str | None = None,
metadata: dict[str, object] | None = None,
now_ms: int | None = None,
) -> ArchiveMarkEnvelope:
"""Insert-or-update one mark assertion with deterministic ``mark_id``."""
"""Insert-or-update one mark assertion with deterministic ``mark_id``.

Message targets carry their resolved owning session in the durable
assertion scope. The indexed message row is rebuildable, so callers that
know the owner should pass it before that row can disappear.
"""
timestamp = now_ms if now_ms is not None else _now_ms()
mark_id = _deterministic_id("mark", target_type, target_id, mark_type)
upsert_assertion(
conn,
assertion_id=assertion_id_for_mark(target_type, target_id, mark_type),
target_ref=f"{target_type}:{target_id}",
scope_ref=f"session:{owner_session_id}" if target_type == "message" and owner_session_id else None,

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

An owner-less re-upsert erases the durable owner scope. Both writers compute scope_ref as an unconditional overwrite. When a message target is re-upserted without owner_session_id, scope_ref becomes None and replaces a previously persisted session:<owner> value. That destroys the durable evidence this PR adds, because the indexed message row is rebuildable and the scope is the surviving authority. ArchiveStore.add_mark and ArchiveStore.save_annotation now reject owner-less message writes, so the facade cannot trigger this, but direct callers of these two functions still can.

  • polylogue/storage/sqlite/archive_tiers/user_write.py#L621-L638: in upsert_mark, preserve the existing scope_ref when owner_session_id is None for a message target, instead of writing None.
  • polylogue/storage/sqlite/archive_tiers/user_write.py#L752-L752: apply the same preservation rule in upsert_annotation.
🛡️ Proposed fix to preserve an already-recorded owner

Add a shared helper:

def _preserved_owner_scope_ref(
    conn: sqlite3.Connection,
    assertion_id: str,
    target_type: str,
    owner_session_id: str | None,
) -> str | None:
    """Return the owner scope for a message target without erasing prior evidence.

    The indexed message row is rebuildable, so a previously recorded owner is
    durable authority. An owner-less re-upsert must not replace it with NULL.
    """
    if target_type != "message":
        return None
    if owner_session_id:
        return f"session:{owner_session_id}"
    existing = read_assertion_envelope(conn, assertion_id)
    return existing.scope_ref if existing is not None else None

Then use it at both sites:

     timestamp = now_ms if now_ms is not None else _now_ms()
     mark_id = _deterministic_id("mark", target_type, target_id, mark_type)
+    assertion_id = assertion_id_for_mark(target_type, target_id, mark_type)
     upsert_assertion(
         conn,
-        assertion_id=assertion_id_for_mark(target_type, target_id, mark_type),
+        assertion_id=assertion_id,
         target_ref=f"{target_type}:{target_id}",
-        scope_ref=f"session:{owner_session_id}" if target_type == "message" and owner_session_id else None,
+        scope_ref=_preserved_owner_scope_ref(conn, assertion_id, target_type, owner_session_id),
         kind=AssertionKind.MARK,
     timestamp = now_ms if now_ms is not None else _now_ms()
     resolved_id = annotation_id or _deterministic_id("annotation", target_type, target_id, body)
+    assertion_id = assertion_id_for_annotation(resolved_id)
     upsert_assertion(
         conn,
-        assertion_id=assertion_id_for_annotation(resolved_id),
+        assertion_id=assertion_id,
         target_ref=f"{target_type}:{target_id}",
-        scope_ref=f"session:{owner_session_id}" if target_type == "message" and owner_session_id else None,
+        scope_ref=_preserved_owner_scope_ref(conn, assertion_id, target_type, owner_session_id),
         kind=AssertionKind.ANNOTATION,
📍 Affects 1 file
  • polylogue/storage/sqlite/archive_tiers/user_write.py#L621-L638 (this comment)
  • polylogue/storage/sqlite/archive_tiers/user_write.py#L752-L752
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@polylogue/storage/sqlite/archive_tiers/user_write.py` around lines 621 - 638,
In polylogue/storage/sqlite/archive_tiers/user_write.py lines 621-638, update
upsert_mark to preserve an existing message owner scope when owner_session_id is
absent; add and reuse a shared helper that returns the explicit session scope or
reads the existing assertion envelope, and use it for scope_ref. Apply the same
helper and preservation rule in upsert_annotation at lines 752-752; non-message
targets should continue producing no owner scope.

Comment on lines +1213 to +1222
result = classify_membership_revisions(
[
revision("raw-z-old", "before", 1),
revision("raw-a-new", "after", 2),
]
)

assert result.accepted_raw_ids == ("raw-a-new",)
assert result.equivalent_raw_ids == ("raw-z-old",)
assert result.ambiguous_raw_ids == ()

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

The raw ids let the pre-existing fallback satisfy the assertion.

sorted over ("raw-z-old", "raw-a-new") already selects raw-a-new. The deterministic raw_id tiebreak at the end of _equal_content_representative therefore produces the same result as the new observed_at_ms tiebreak. The test passes even if the observed_at_ms branch is removed.

Choose raw ids whose alphabetical order favors the older revision. Then only the observation-order rule can produce the expected accepted id.

💚 Proposed fix to make the test discriminate the new tiebreak
     result = classify_membership_revisions(
         [
-            revision("raw-z-old", "before", 1),
-            revision("raw-a-new", "after", 2),
+            revision("raw-a-old", "before", 1),
+            revision("raw-z-new", "after", 2),
         ]
     )
 
-    assert result.accepted_raw_ids == ("raw-a-new",)
-    assert result.equivalent_raw_ids == ("raw-z-old",)
+    assert result.accepted_raw_ids == ("raw-z-new",)
+    assert result.equivalent_raw_ids == ("raw-a-old",)
     assert result.ambiguous_raw_ids == ()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
result = classify_membership_revisions(
[
revision("raw-z-old", "before", 1),
revision("raw-a-new", "after", 2),
]
)
assert result.accepted_raw_ids == ("raw-a-new",)
assert result.equivalent_raw_ids == ("raw-z-old",)
assert result.ambiguous_raw_ids == ()
result = classify_membership_revisions(
[
revision("raw-a-old", "before", 1),
revision("raw-z-new", "after", 2),
]
)
assert result.accepted_raw_ids == ("raw-z-new",)
assert result.equivalent_raw_ids == ("raw-a-old",)
assert result.ambiguous_raw_ids == ()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/archive/test_session_revision_membership.py` around lines 1213 -
1222, Update the test case for classify_membership_revisions to use raw IDs
whose alphabetical order favors the older revision, while keeping the newer
revision’s observed_at_ms value later. Preserve the expected accepted_raw_ids
result so the assertion can only pass when _equal_content_representative applies
the observed_at_ms tiebreak.

Comment on lines +62 to +70
assert row["bead_status"] == "in_progress"
assert row["residual_successor"] == {
"bead_id": "polylogue-message-owner-scope-backfill",
"kind": "named-child-bead",
}
assert set(cast(list[str], graph_entry["child_bead_ids"])) == {
"polylogue-xselt",
"polylogue-message-owner-scope-backfill",
}

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Align the assertions with the updated ledger and campaign graph.

Line 62 expects in_progress, but docs/plans/reindex-incident-coverage.json now sets polylogue-slshy to closed. Lines 63-70 expect the old residual successor and child set. The ledger and graph now specify only polylogue-xselt. This test will fail.

Proposed fix
-    assert row["bead_status"] == "in_progress"
+    assert row["bead_status"] == "closed"
     assert row["residual_successor"] == {
-        "bead_id": "polylogue-message-owner-scope-backfill",
+        "bead_id": "polylogue-xselt",
         "kind": "named-child-bead",
     }
     assert set(cast(list[str], graph_entry["child_bead_ids"])) == {
         "polylogue-xselt",
-        "polylogue-message-owner-scope-backfill",
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
assert row["bead_status"] == "in_progress"
assert row["residual_successor"] == {
"bead_id": "polylogue-message-owner-scope-backfill",
"kind": "named-child-bead",
}
assert set(cast(list[str], graph_entry["child_bead_ids"])) == {
"polylogue-xselt",
"polylogue-message-owner-scope-backfill",
}
assert row["bead_status"] == "closed"
assert row["residual_successor"] == {
"bead_id": "polylogue-xselt",
"kind": "named-child-bead",
}
assert set(cast(list[str], graph_entry["child_bead_ids"])) == {
"polylogue-xselt",
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/devtools/test_incident_coverage_ledger.py` around lines 62 - 70,
Update the assertions in the incident coverage test to match the current ledger:
expect polylogue-slshy’s bead_status to be closed, expect residual_successor to
reference only polylogue-xselt, and assert the child_bead_ids set contains only
polylogue-xselt.

assert message.provider_message_id.startswith("synthetic-")
assert message.provider_message_id == ""

def test_reasoning_reordering_keeps_synthetic_revision_identity(self) -> None:

ghost Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Rename the test; it no longer covers a synthetic identity.

This PR removes the synthetic-* provider message id from the Codex reasoning path, as line 679 now asserts. The test body compares message_contents across a reorder, which exercises the content-derived revision anchor in _message_revision_match_id, not a synthetic id. The current name states the opposite of the new behavior.

♻️ Proposed rename
-    def test_reasoning_reordering_keeps_synthetic_revision_identity(self) -> None:
+    def test_reasoning_reordering_keeps_content_anchored_revision_identity(self) -> None:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def test_reasoning_reordering_keeps_synthetic_revision_identity(self) -> None:
def test_reasoning_reordering_keeps_content_anchored_revision_identity(self) -> None:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/sources/test_parsers_codex.py` at line 681, Rename
test_reasoning_reordering_keeps_synthetic_revision_identity to reflect that it
verifies content-derived revision identity across message reordering, removing
the obsolete synthetic-identity wording while leaving the test body unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant