Skip to content

feat: title model selection evaluations by session and name mentions in titles - #2562

Merged
zfy0701 merged 3 commits into
mainfrom
claude/model-selection-evaluation-title
Sep 27, 2026
Merged

zfy0701 merged 3 commits into
mainfrom
claude/model-selection-evaluation-title

Conversation

@zfy0701

@zfy0701 zfy0701 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Two gaps in the evaluation titles from #2552, both visible on the Decision page's places card from #2561:

  1. Model selection rows had no title. A Decision used only for an Agent's model selection showed rows "titled" by the chosen runtime and model (claude-acp · opus[1m]), which says nothing about what was judged.
  2. Chat titles quoted Slack mentions raw, so a row read <@U0123ABC> hi.

Model selection: titled by the session

  • protocol: DecisionModelEvaluationRecord gains the same nullable title as the other records.
  • daemon: the model selection list and detail title each row from its own session record, naming the title's <@U…> mentions from the display-name cache as the daemon's session reader does (one name lookup per page; a session the daemon no longer holds leaves the row untitled). Titling happens before the 32 KiB page cap is applied. The stored summary still has no title.
  • control plane: returns a row only when the caller can view its session, as before. The daemon's title wins because it names mentions (SessionMeta.title keeps a first-message title's mentions raw); when the daemon has none because a session retention shorter than the summaries' seven days removed its session row, the CP uses its own SessionMeta.title, which it already loaded for the visibility check.
  • Why the session and not the frozen input: a model selection's input expires after 24 hours or 20 newer choices, while every model selection belongs to a session whose title persists. For a code-host event that title is already PR #42: ….
  • web: the Decision page shows the session title as the row's first line and moves the chosen target to the source line (review-bot → claude-acp · opus[1m]).

Mentions named as session titles name them

  • summaryRow now replaces <@U…> mentions with @<display name> from the daemon's display_names cache before the 80-character cut, using the same mentionedUserIds / substituteUserMentions helpers the session title path uses. An unknown id stays raw, as it does there.
  • One getDisplayNames query per page (and per detail read), collecting ids from the page's stored inputs.

Docs

decisions.md §9.5 says how model selection rows are titled and that mentions are named.

Testing

  • daemon: a model selection row's session title has its known mention named in the list and the detail, and a session the daemon no longer holds leaves the row untitled; a known chat mention is named and an unknown one kept in the list and the detail; the Decision and model suites (306)
  • control plane integration: the model selection list and detail stay scoped to visible sessions, pass the daemon's title through, and fall back to the session's CP title when the daemon sends none
  • protocol, web Decision component suites, typecheck and lint for all four packages

🤖 Generated with Claude Code . Claude Opus 5.5

…in titles

A Decision used only for an Agent's model selection showed rows titled
by the chosen runtime and model, because model selection records carried
no title. Their input expires within a day, so the daemon cannot derive
one for long; instead the CP fills each row's title from its session's
title, from the session record it already reads to check that the caller
may view that session. The Decision page shows the chosen runtime and
model after the place instead.

Titles derived from a chat message quoted Slack user mentions raw
(<@U…>). They now name them from the daemon's stored display names
before the 80-character cut, as session titles do, with one lookup per
page; an unknown id stays raw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new model-selection title path still displays raw Slack mentions for sessions opened by a mention. For example, a first message <@U0KNOWN1> hi is stored as the session title and synced to SessionMeta unchanged; mention substitution happens later in the daemon’s session-list reader. This route copies SessionMeta.title into the evaluation, so the Decision page shows <@U0KNOWN1> hi even when the display-name cache knows that user. Please resolve mentions in the title supplied to model-selection evaluations, as the other evaluation summaries now do.

sent by review-bot (Codex · gpt-6-sol) · open in session

Comment thread packages/control-plane/src/http/routes/agent-model-evaluations.ts
@zfy0701
zfy0701 enabled auto-merge (squash) September 27, 2026 01:17
@zfy0701
zfy0701 disabled auto-merge September 27, 2026 01:17
…emon

The CP copied the session title from its session record, which keeps a
title derived from a first message such as <@U…> hi unchanged; names are
substituted only when the daemon reads sessions. The daemon now titles
each model selection row from its own session record and names the
title's mentions from its display names, as its session reader does, and
the CP passes the row through after its session visibility check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The raw Slack mention issue is fixed while the daemon still has the session row. The revised title source does not last as long as the evaluation summary: sessions.retention supports 1d and deletes local session rows after that window, while decision_model_evaluation summaries remain for seven days. With a 1-day setting, visible model-selection rows become untitled for their remaining six days even though the control plane retains the session metadata and title. Please keep a title source available for the full evaluation-summary lifetime.

sent by review-bot (Codex · gpt-6-sol) · open in session

Comment thread packages/daemon/src/decisions/model-evaluations.ts
…ection row

A deployment can keep daemon session rows for as little as a day while
model selection summaries last seven, so the daemon's session title can
disappear from a row that is still listed. The CP now uses its own copy
of the session title, already loaded for the visibility check, whenever
the daemon sends none; the daemon's title, which names mentions, still
wins while its session row exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The model-selection list and detail keep the daemon’s mention-resolved title while its session record exists, then fall back to the control plane’s retained session title after local session retention removes that record. Both paths still return rows only for sessions the caller may view. The fallback’s raw mentions after purge are documented and are a non-blocking presentation limit.

sent by review-bot (Codex · gpt-6-sol) · open in session

@zfy0701
zfy0701 merged commit 8311f30 into main Sep 27, 2026
15 checks passed
@zfy0701
zfy0701 deleted the claude/model-selection-evaluation-title branch September 27, 2026 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant