Skip to content

prompts: calibrate network hardware findings - #424

Open
OllieinCanada wants to merge 1 commit into
sashiko-dev:mainfrom
OllieinCanada:fix/99-netdev-hardware-severity
Open

prompts: calibrate network hardware findings#424
OllieinCanada wants to merge 1 commit into
sashiko-dev:mainfrom
OllieinCanada:fix/99-netdev-hardware-severity

Conversation

@OllieinCanada

@OllieinCanada OllieinCanada commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Fixes #99

Summary

Add targeted network-driver guidance for three hardware-error false-positive patterns reported in #99 and maintainer feedback.

Evidence

  • a bnge cleanup path where device ownership had not transferred before driver registration
  • an Intel configuration-register race claim with no actual concurrent writer
  • DMA teardown warnings after a device stop failure or timeout, where leaking the buffer indefinitely is not a practical fix

No aggregate evaluation data is available, so this change stays limited to those concrete patterns.

Changes

  • check for an actual writer or documented hardware behavior before treating configuration-register concurrency as established
  • verify that ownership transfer succeeded before rejecting host-side cleanup
  • check available reset, quiesce, and documented teardown steps after a failed stop command without recommending an indefinite DMA-buffer leak
  • rate severity from demonstrated impact rather than confidence in the premise

Validation

  • exact one-file cumulative diff
  • git diff --check
  • signed-off commit uses the contributor's real DCO identity
  • no model-backed regression or paid API call was used

@rgushchin

Copy link
Copy Markdown
Member

Same here: do you have any examples/data on where/how it helps?
Also, can you, please, use your real name and email for the SOB line?
Thank you!

cc @kuba-moo Jakub, does this change look good to you?

@kuba-moo

Copy link
Copy Markdown
Contributor

Looks too strong for me.
The "imagined behavior" of LLMs is quite often correct, I'd drop the first section completely.
Concurrency thing is good but again probably phrased stronger that I'd phrase it.
Cap to medium is good.

FWIW my main annoyance with the HW behavior right now is models reporting that "device may still be writing to the DMA buffer" after a failure to stop or a timeout. Entirely true, but the only choices are leak memory or ignore, and the latter is understood to be the best practice. That's something that belongs in review prompts tho.

Sidetrack, Roman, I was wondering about practicality of separating the strength of the finding from the severity. IOW confidence vs severity. Have you toyed with that?

@rgushchin

Copy link
Copy Markdown
Member

@kuba-moo I did, my impression is that whatever LLMs are talking about their confidence is almost completely random. Maybe only something like "drop the finding unless you can 100% show all code paths leading to the described behavior" works, but it's prone to drop real issues too. Maybe some newer models are better, Idk.
My understanding is that confidence is hard and it requires running some sort of multiple parallel thinking processes in "the head" and then compare results.
Currently severity is only severity, IMO it should not depend on confidence level at all. I don't like "it's medium unless you are 100% sure".

@OllieinCanada
OllieinCanada marked this pull request as draft August 14, 2026 15:31
@OllieinCanada

Copy link
Copy Markdown
Contributor Author

The concrete examples are the two false positives recorded in #99: the bnge path where ownership had not transferred before driver registration, and the Intel path that assumed a configuration register could change concurrently without an actual writer. Jakub's DMA-after-stop-timeout example here is a third useful case. I don't have aggregate stats, so I won't overstate the evidence. I'll revise this to drop the broad imagined-hardware section, soften the concurrency wording, add the DMA cleanup case, and keep severity separate from confidence.

Signed-off-by: Oliver Slapinski <olliefromcanada@gmail.com>
@OllieinCanada
OllieinCanada force-pushed the fix/99-netdev-hardware-severity branch from bce191f to da5ce04 Compare August 14, 2026 18:14
@OllieinCanada
OllieinCanada marked this pull request as ready for review August 14, 2026 18:20
@OllieinCanada

Copy link
Copy Markdown
Contributor Author

Updated the prompt based on the discussion: removed the broad imagined-hardware rule, softened the register-concurrency guidance, added the DMA teardown case, and separated severity from confidence. I also corrected the author and Signed-off-by identity. All checks pass, and the PR is ready for another look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Annotate or downgrade all reports related to device misbehavior

3 participants