Make v2-primary reads resolve legacy conversation IDs, name direct threads, and surface projection lag (#155)

[MaxGhenis]

Closes #155.

The report

A Signal 1:1 thread readable on 7/25 via
GET /api/conversations/signal%3A%2B1XXXXXXXXXX/messages returned [] on 7/26
after two app restarts, and the thread was absent from /api/conversations
while /api/status insisted Signal was connected: true, paired: true. The MCP
get_conversation path was equally empty.

No messages were lost. Four distinct defects, all consequences of the
v2-primary cutover, made an intact thread unaddressable.

What was wrong

1. v2 reads only ever resolved v2 primary keys. Cutover re-keys every
conversation to v2keys.DeriveID(...) (a 32-hex hash) and preserves the legacy
id as conversations.remote_conversation_id. v2read.Source looked up only the
primary key, so every stored pre-cutover reference — the exact string agents and
notes had been carrying for days — silently read as an empty thread the moment a
restart flipped the app to v2-primary. Reads now fall back to a
remote_conversation_id lookup across accounts (deterministic: most recently
active wins, so a number that threads on two platforms can't flap), and the
returned DTO always carries the canonical v2 id.

2. Direct conversations had no name. Only the Google decoder emits
ConversationEvent frames; Signal and WhatsApp conversations are minted from
message frames, which carry no title. Legacy named 1:1s after the remote peer,
so post-cutover Signal threads listed as blank rows. mapConversation now names
direct threads after the peer, falling back to the peer's address.

3. Direct conversations had no participants. Same root cause — message
frames carry no roster, so 1:1 threads read Participants: "[]". The projector
now links the peer when projecting a direct-conversation message (inbound sender
identity; for outgoing-only threads, the peer named by an address-shaped remote
id), and a daemon-startup sweep backfills rows that predate the fix.

4. Post-cutover Signal/WhatsApp groups were stored as direct. Message
events carry no kind and the projector defaulted all of them, so groups
mis-render as 1:1s — and it would have let the peer repair in (3) attach whoever
spoke first as a group's sole counterparty. Kind is now inferred from the remote
id where that is authoritative (signal-group:, @g.us, @broadcast), never
guessed for opaque Google thread ids, and the sweep corrects mislabeled rows
without touching their rosters.

Plus the reason nobody noticed for two days: /api/status freshness measures
only the active read source. Running v2-primary, a stalled v2 ingest projection
is invisible there — the legacy path kept ingesting from Thu 7/23 to Sat 7/25
while readers saw nothing new, and every platform still reported
behind_days: 0. Each platform entry now also carries legacy_latest_ms,
projection_lag_ms, and projection_stalled, plus a top-level
projection_stalled.

The "placeholder timestamps" were a reader bug, not a data bug

Reported as "every message shows Wed 19:00". The legacy DTO marshals
TimestampMS (Go field name, no tag); the reading script looked for
timestamp_ms/TimestampMs, got 0, and fromtimestamp(0) renders as
1969-12-31 19:00 ET. Stored timestamps were correct the whole time. The web UI
reads TimestampMS in 15 places, so renaming it would break the app — the field
names and this failure signature are documented in the runbook instead.

Verification

Unit and integration tests, plus the Signal worker harness driving real
signal-cli envelopes end to end: peer linked for an inbound 1:1, peer derived
from the remote id for an outgoing-only thread, no participant synthesized for a
group, opaque Google ids left alone, alias resolution for legacy/v2/group keys,
deterministic cross-account resolution, projection-lag reporting, and
idempotence of the sweep. GOWORK=off go test ./... is green.

Also validated against a read-only copy of a live store (temp harness,
removed before commit — the app was never touched):

• signal:+1XXXXXXXXXX → resolves to its v2 hash, correct TimestampMS, named
  after the peer once the sweep runs.
• Sweep on that copy: 17 bare direct conversations → 12 linked, 4 mislabeled
  group kinds corrected, 1 unattributed opaque thread left alone, second run a
  no-op.
• A real signal-group: row stored as direct flipped to group: true.
• An unknown key still returns "no rows" rather than matching something else.

That exercise also caught a fabrication risk in my own sweep: for opaque Google
thread ids, group-ness is unknown, so linking "the latest inbound sender" could
render a group as a 1:1. The sweep now requires the only available evidence —
exactly one distinct inbound sender — before naming such a thread's peer.

Live-app note

Per the working agreement: the running app was not touched — no restart, no
stop, no write to the live data dir. All validation ran against copies in a temp
dir. The participant/kind sweep runs on the next ordinary app start; nothing here
requires a manual step.

Still open (follow-up, not in this PR)

The v2 store is missing the Signal messages from the stall window that only
legacy has (~98 in the thread that prompted this, Thu 20:17 → Sat 15:00). Reads
are durable now, but that history is genuinely absent from the store the app
reads. An openmessage v2 reconcile-signal maintenance command is in progress on
a parallel track; key parity is already verified — legacy source_id is
byte-identical to what the live decoder derives for both directions
(v2keys.SignalIncomingSourceID inbound, v2keys.SignalLocalAlias outbound), so
reconciliation can dedupe exactly rather than duplicate.

Note: this branch also contains one commit (PROGRESS.md) from the parallel
agent working on that command.

🤖 Generated with Claude Code

[MaxGhenis]

Scope note. A parallel agent committed the in-progress reconciliation package onto this branch mid-review. I reverted it here (d47a134 → revert) so this PR stays what its description says: read durability only, 15 files. The reconciliation work is preserved on wip/signal-reconcile and is not proposed for merge — it mutates the message store and needs its own review, plus a dry run against a copy before any real run.

Worth recording from that work, because it is the sharp edge of any future reconciliation: v2 and legacy disagree on outgoing Signal message keys. The live decoder keys a sync-sent message by its bare send timestamp (1785072320483); legacy and R2 migration key the same message local:<sha1> (v2keys.SignalLocalAlias). Incoming messages agree (both use SignalIncomingSourceID), which makes the disagreement easy to miss. Deduping on the legacy key alone misses every outgoing message the live decoder already projected — 50 of 50 on the thread I checked — and re-imports each as a duplicate. Both forms must be checked. That contract has regression tests on the WIP branch.

[MaxGhenis]

Adversarial review (gpt-5.6-sol, ultra reasoning) — REQUEST-CHANGES, not safe to redeploy yet. Five findings:

1. High — WhatsApp direct threads gain duplicate identities/participants. Outgoing direct messages take the remote-ID fallback (worker.go ~1166) and extract the literal …@s.whatsapp.net JID (participants.go ~125), while inbound decoding canonicalizes the same peer to E.164 (whatsappdecoder.go ~490) — two distinct identities both get linked, and naming can pick the raw JID. New integration tests cover Signal only.

2. High — opaque Google threads still get guessed as direct. A message arriving before/without its conversation frame defaults the thread to direct (worker.go ~824), then links the inbound sender as peer (worker.go ~1166) — despite the classifier refusing to infer Google shape (participants.go ~104). Frames append independently and failures are nonfatal, so the ordering is realistic. The sweep's senders > 1 guard is insufficient (one observed speaker ≠ proof of 1:1), and the additive design (participants_ensure.go ~100) makes a wrong link permanent.

3. High — outgoing sends mask inbound projection stalls. addProjectionLag compares overall LatestMS only (api.go ~3758/3789); an optimistic v2 outbox insert zeroes the computed lag while LatestRecvMS is days behind. Include the received-message gap. Also: a missing v2 platform reports projection_lag_ms as the raw Unix timestamp, not a duration (api.go ~3775).

4. High — legacy-ID support misses thread-scoped search. SearchFilter.ConversationID goes into an exact v2-PK comparison (v2read/search.go ~20), so /api/conversations/{legacy-id}/search returns empty post-cutover.

5. Medium — legacy deep links bypass the resolver. The UI exact-matches ?conversation= against canonical list IDs (index.html ~14037); old bookmarks strand on no selected thread.

Plus: the alias resolver suppresses account/lookup errors as "no match" (alias.go ~32); cross-account selection intentionally rebinds on activity (consumers must pin the returned canonical id) but the test doesn't prove recency-wins (freshest match is also last in account order) and lacks a tie case. Missing coverage: WhatsApp out/in sequence, Google message-without-conversation, post-sweep live message, outbound-masked stall, alias search/deep links, collision tie.

Full review transcript available locally (sol-review-156). The four-defect diagnosis in the PR body stands — these findings are about the fix's edge cases, not the diagnosis.