Skip to content

fix(memory): harden preference secret classifier - #79

Open
senoldogann wants to merge 20 commits into
mainfrom
fix/preference-secret-classifier
Open

senoldogann wants to merge 20 commits into
mainfrom
fix/preference-secret-classifier

Conversation

@senoldogann

@senoldogann senoldogann commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Fixes #77.

Classifier policy

  • normalize named credential labels (password, passwd, api key, compact apikey, api_key, api-key, bounded spaces/tabs, token, secret)
  • : / = credential assignments fail closed
  • whitespace-only alphabetic credential values are sensitive
  • ambiguous copular/grammar-only followers fail closed, including present/past (is, was, were), perfect-tense terminal been, existing policy connectors (should, must), and bounded modal + be forms (will, would, can, could, may, might, shall)
  • punctuation is not credential-complexity evidence
  • ordinary credential-related prose survives only with positive semantic context such as password-manager, token-budget, secret-rotation, or API-key-rotation language
  • safe prose followers cannot hide immediate or later same-clause delimiter/copular credential assignments
  • full untruncated clauses are screened before bounding
  • extraction and persistence use the same typed classifier
  • classifier diagnostics do not retain or echo rejected secret values

TDD evidence

  • Initial RED 55743d44 / CI #294: 16 failed, 1097 passed, 2 skipped.
  • Copular self-review RED ac269556 / CI #296: 5 failed, 1113 passed, 2 skipped.
  • Compact apikey review RED a56f1089 / CI #298: 3 failed, 1118 passed, 2 skipped; fixed by 7abc8e1382.
  • Safe-follower delimiter review RED dc1f28eb / CI #306: 7 failed, 1121 passed, 2 skipped; fixed by c0921df782.
  • Safe-follower copular composition RED 174796df / CI #314: 7 failed, 1128 passed, 2 skipped; fixed by 490f7b8aa1ffd73a49c32a42c3e1c70eaca000de.
  • Later safe-follower assignment RED 58d00e7d / CI #324: 5 failed, 1135 passed, 2 skipped; fixed by f1d26ce6a64dc469ab97be4cf2dd063578b06015.
  • Past-tense safe-follower assignment RED dd0f9e8c / CI #330: 6 failed, 1140 passed, 2 skipped; fixed by 00e6fc27c0f138f7469f7f82acd71eb717b7564d.
  • Modal safe-follower assignment RED 5d2878a2 / CI #335: 9 failed, 1146 passed, 2 skipped; fixed by 55eab97465bd8373082d04efae81f1096eb1c331.
  • Perfect-tense safe-follower assignment RED 29d67d3f / CI #341: 12 failed, 1155 passed, 2 skipped; fixed by df3f42586dc7fbd046df8c07927a7076f2513c7d, with trailing-newline cleanup 2a6728fc2e028422c68f0fdfdc69927d8acd1342.
  • Final candidate CI #343 on exact head 2a6728fc2e028422c68f0fdfdc69927d8acd1342: 1167 passed, 2 skipped; Ruff clean; Pyright 0 errors / 0 warnings / 0 informations; Linux Rust GREEN; macOS real-backend Rust GREEN.

Merge discipline

Do not merge until automated review has completed on the exact final head SHA and every inline finding has been reproduced/fixed or concretely closed. After merge, observe main CI independently before #78 / PR D work proceeds.

@senoldogann
senoldogann marked this pull request as ready for review September 8, 2026 09:02

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: caf97d75bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a56f1089fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7abc8e1382

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0921df782

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

Copy link
Copy Markdown
Owner Author

Self-review found and reproduced one additional compositional credential bypass after the previous review: an allowlisted follower could hide a copular value, e.g. password policy is swordfish. Test-only 174796df31fd1c73443a72d687efb2f174b66e80 produced CI #314 with exactly 7 failures / 1128 passed / 2 skipped (five classifier cases, structured extraction, direct store) while Ruff, Pyright and both Rust jobs stayed green. Minimal production fix 490f7b8aa1ffd73a49c32a42c3e1c70eaca000de adds a bounded is|should|must trailing-value check without changing the safe-prose allowlist. CI #318 is fully green: 1135 passed / 2 skipped, Ruff clean, Pyright 0 errors/warnings/informations, Linux Rust green, macOS real-backend Rust green.

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 490f7b8aa1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

Copy link
Copy Markdown
Owner Author

@codex address the current test-only RED on head 58d00e7d54bd84650b781d58fcac2857284132b6. Keep the existing tests unchanged. Root cause: after an allowlisted credential prose follower, the classifier only checks delimiter/copular assignments at the start of the remaining tail, so later same-clause forms like password policy value: swordfish, password manager entry = swordfish, and token budget value is abcdefghijklmnop escape. Make the smallest production-only change in src/computeruse/memory/preferences.py so safe-prose exemption is withheld when the remaining same-clause tail contains a later :/= assignment or is|should|must value assignment. Preserve current safe prose behavior and the existing compact apikey, immediate delimiter, and immediate copular fixes. Do not broaden scope or refactor unrelated code.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f1d26ce6a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00e6fc27c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d2878a24d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/computeruse/memory/preferences.py

Copy link
Copy Markdown
Owner Author

@codex review

Copy link
Copy Markdown
Owner Author

@codex review

Please review the exact current head 2a6728fc2e028422c68f0fdfdc69927d8acd1342. CI #343 is green with 1167 passed / 2 skipped, Ruff clean, Pyright 0 errors/warnings/informations, Linux Rust green, and macOS real-backend Rust green. All prior inline findings have been reproduced with RED tests, fixed, and resolved. Please focus on any remaining P1/P2 secret-classification bypasses or false-positive regressions before merge.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a6728fc2e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +68 to +71
r"(?i)[ \t]+(?:"
r"is|was|were|been|should|must|"
r"(?:will|would|can|could|may|might|shall)[ \t]+be"
r")(?:[ \t]+|[ \t]*[:=][ \t]*)(?P<value>\S+)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Recognize word-form credential assignments

Fresh evidence beyond the resolved copula cases is that an explicit assignment such as password manager entry equals swordfish still bypasses classification: manager is allowlisted, while neither assignment regex recognizes equals, so structured extraction and direct PreferenceStore.record calls can persist the secret. Treat word-form assignment operators such as equals as assignments before applying the safe-prose exemption.

AGENTS.md reference: AGENTS.md:L33-L36

Useful? React with 👍 / 👎.

Comment on lines +320 to +322
introduces_assignment = (
raw_follower.endswith((":", "=")) and bool(trailing.strip())
) or _DELIMITED_CREDENTIAL_VALUE_RE.search(trailing) is not None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude URL schemes from trailing assignment detection

When an ordinary preference mentions a URL after safe credential prose—for example preference: security=use the password manager workflow in https://vault.example—the unrestricted delimiter search treats the colon in https: as a credential assignment and rejects the preference. This regresses both explicit extraction and direct store writes for non-secret security preferences containing URLs; restrict trailing delimiter detection to assignment positions or explicitly exclude URI schemes.

AGENTS.md reference: AGENTS.md:L33-L36

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(memory): harden preference secret classifier

1 participant