Skip to content

Cover the live L2 path against a real server - #241

Merged
evkir merged 1 commit into
mainfrom
w3/detector-eval-l2-coverage
Aug 28, 2026
Merged

Cover the live L2 path against a real server#241
evkir merged 1 commit into
mainfrom
w3/detector-eval-l2-coverage

Conversation

@evkir

@evkir evkir commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Coverage reported eight lines missing in detector_eval: the --l2 branch and the block that writes a recording. They are the only branches of the command that talk to a model, so nothing reached them, and mocking them would have verified a stand-in for the code rather than the code.

A socket that speaks ollama's shape reaches them instead. The flag, the recording wrapper, the default transport and the file written all run as they do live, and the test asserts the property the committed artifact's gate depends on: the live run, the bytes it wrote and a replayed run agree.

That required the address to become an option, which is not a concession to the test. ollama does not have to sit on this host, and hard-coding that it does was a defect the missing coverage happened to surface.

What this changes

How it was measured

Checklist

  • ruff format --check cyberai/ tests/ and ruff check cyberai/ tests/ pass
  • pytest -W ignore::DeprecationWarning -m "not slow and not smoke" passes
  • New behaviour is covered by a test that fails without the change
  • I have read CLA.md and I hereby sign the CLA

Coverage reported eight lines missing in detector_eval: the --l2 branch and
the block that writes a recording. They are the only branches of the command
that talk to a model, so nothing reached them, and mocking them would have
verified a stand-in for the code rather than the code.

A socket that speaks ollama's shape reaches them instead. The flag, the
recording wrapper, the default transport and the file written all run as
they do live, and the test asserts the property the committed artifact's
gate depends on: the live run, the bytes it wrote and a replayed run agree.

That required the address to become an option, which is not a concession to
the test. ollama does not have to sit on this host, and hard-coding that it
does was a defect the missing coverage happened to surface.
@evkir
evkir merged commit 8664576 into main Aug 28, 2026
7 checks passed
@evkir
evkir deleted the w3/detector-eval-l2-coverage branch August 28, 2026 19:06
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants