fix: corroborate ack-channel correction detection (v1.3.3) - #1
Merged
Conversation
…lse positives (v1.3.3) The Stop-hook auto-proposer treated the assistant's acknowledgment language (channel 2: "you're right", "my mistake", "i was wrong") as sufficient to arm a correction on its own. But that vocabulary also covers ordinary agreement and Claude self-correcting its OWN prior claim in conversation — the dominant false positive. A neutral user instruction answered apologetically would fire a rule nudge with no actual user correction. Channel 2 is now CONFIRMING, not self-sufficient: an assistant ack counts as a correction only when the user's message also shows doubt (a question mark, a hard CORRECTION_PATTERN, or a soft DOUBT_LEXICON term). Channel 1 (explicit user phrasing) still arms a correction alone at the unchanged low bar, so genuine low-tool corrections (e.g. the web-search-for-latest-versions rule) are preserved. Deliberately did NOT add a score/edit gate on rule capture: verified it would suppress real corrections that do no edits and score 0 (web-search confirmations). - add DOUBT_LEXICON + _user_shows_doubt() (opening-window scan) - score_turn: ack_is_correction = correction_ack and user_doubt - expose user_doubt in the signals dict for observability - 4 regression tests; 59/59 green Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Stop-hook auto-proposer (
scripts/auto_propose.py) over-fires rule-capture nudges. Its channel 2 (assistant acknowledgment) treats Claude's own phrasing — "you're right", "my mistake", "i was wrong" — as enough to arm a correction on its own.But that vocabulary also covers ordinary agreement and Claude self-correcting its own prior claim in conversation. The dominant false positive: a neutral user instruction answered apologetically fires a rule nudge with no actual user correction. Observed live (
auto_propose.jsonl):correction_user: false, correction_ack: trueon turns where the user only gave a directive.Fix
Channel 2 becomes confirming, not self-sufficient: an assistant ack counts as a correction only when the user's message also shows doubt — a
?, a hardCORRECTION_PATTERN, or a softDOUBT_LEXICONterm.Channel 1 (explicit user phrasing) still arms a correction alone at the unchanged low bar, so genuine low-tool corrections (e.g. the
web-search-for-latest-versionsrule — web searches, no edits, score 0) are preserved.Deliberately not done: a score/edit gate on rule capture. Verified it would suppress real corrections that do no edits and score 0 (web-search confirmations) — a degradation.
Changes
DOUBT_LEXICON+_user_shows_doubt()(opening-window scan, mirrors existing guards)score_turn:ack_is_correction = correction_ack and user_doubtuser_doubtin the signals dict for observabilityplugin.json1.3.2 → 1.3.3Tests
4 new regression tests; 59/59 green. No existing assertion changed (no true-positive regressions):
test_ack_alone_without_user_doubt_does_not_fire(the regression)test_assistant_self_correction_in_meta_discussion_silenttest_ack_with_user_question_mark_still_fires(preserves channel 2)test_ack_with_soft_doubt_no_question_mark_fires🤖 Generated with Claude Code