prompts: calibrate network hardware findings - #424
Conversation
|
Same here: do you have any examples/data on where/how it helps? cc @kuba-moo Jakub, does this change look good to you? |
|
Looks too strong for me. FWIW my main annoyance with the HW behavior right now is models reporting that "device may still be writing to the DMA buffer" after a failure to stop or a timeout. Entirely true, but the only choices are leak memory or ignore, and the latter is understood to be the best practice. That's something that belongs in review prompts tho. Sidetrack, Roman, I was wondering about practicality of separating the strength of the finding from the severity. IOW confidence vs severity. Have you toyed with that? |
|
@kuba-moo I did, my impression is that whatever LLMs are talking about their confidence is almost completely random. Maybe only something like "drop the finding unless you can 100% show all code paths leading to the described behavior" works, but it's prone to drop real issues too. Maybe some newer models are better, Idk. |
|
The concrete examples are the two false positives recorded in #99: the bnge path where ownership had not transferred before driver registration, and the Intel path that assumed a configuration register could change concurrently without an actual writer. Jakub's DMA-after-stop-timeout example here is a third useful case. I don't have aggregate stats, so I won't overstate the evidence. I'll revise this to drop the broad imagined-hardware section, soften the concurrency wording, add the DMA cleanup case, and keep severity separate from confidence. |
Signed-off-by: Oliver Slapinski <olliefromcanada@gmail.com>
bce191f to
da5ce04
Compare
|
Updated the prompt based on the discussion: removed the broad imagined-hardware rule, softened the register-concurrency guidance, added the DMA teardown case, and separated severity from confidence. I also corrected the author and Signed-off-by identity. All checks pass, and the PR is ready for another look. |
Fixes #99
Summary
Add targeted network-driver guidance for three hardware-error false-positive patterns reported in #99 and maintainer feedback.
Evidence
No aggregate evaluation data is available, so this change stays limited to those concrete patterns.
Changes
Validation
git diff --check