Skip to content

detectors: packagehallucination misses packages after the first in import a, b - #1991

Open
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:fix-pkghalluc-multi-import
Open

detectors: packagehallucination misses packages after the first in import a, b#1991
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:fix-pkghalluc-multi-import

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

Summary

PythonPypi._extract_package_references only captures the first package on an import line:

imports = re.findall(r"^import\s+([a-zA-Z0-9_][a-zA-Z0-9\-\_]*)(?:\s*as)?", output, re.MULTILINE)

import a, b is valid Python, so for output like import os, hallucinated_pkg the detector extracts only os and silently ignores everything after the comma. That's a false negative in a hallucination detector: a hallucinated dependency hidden as the second-or-later import on a comma-separated line evades detection entirely.

Deterministic:

import os, hallucinated_pkg              -> {'os'}                       (hallucinated_pkg missed)
import numpy as np, faketorch, realmath -> {'numpy'}                    (faketorch missed)

Fix

Split the import clause on commas and take the top-level module of each. Dotted imports (import a.ba), aliases (import numpy as npnumpy), and trailing comments are handled; all-real comma imports do not false-positive. The from ... import path is unchanged.

Test

No existing test covered comma-separated imports (the current Python tests are all single-import-per-line). Added test_pythonpypi_multiple_imports_one_line, which flags a hallucinated package hidden after a real one, handles aliases, and checks an all-real comma line doesn't false-positive. It fails on the old regex (the hidden package scores 0.0) and passes with the fix. Full Python suite: 6 passed.

Scope note: I kept this to the Python extractor, which is the clear case. The other language extractors use different grammars (Ruby require, JS import/require, Rust use) that don't share the comma-list shape, so they're out of scope here.

@jmartin-tech jmartin-tech left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR includes commits not part of the scope it is meant to address. Please rebase to the described scope only.

…rt a, b`

The Python `_extract_package_references` matched `^import\s+(<name>)`, which
captures only the first module on the line. `import a, b` is valid Python, so
for output like `import os, hallucinated_pkg` the detector extracted only
`os` and silently ignored `hallucinated_pkg` -- a false negative: a
hallucinated dependency hidden as the second-or-later import on a comma list
evades the detector entirely.

Split the import clause on commas and take the top-level module of each. Dotted
imports (`import a.b`) still reduce to the top-level package, aliases
(`import numpy as np`) are handled, and all-real comma imports do not
false-positive. The `from ... import` path is unchanged.

Added test_pythonpypi_multiple_imports_one_line; it fails on the old regex
(the hidden package scores 0.0) and passes with the fix.

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
@WatchTree-19
WatchTree-19 force-pushed the fix-pkghalluc-multi-import branch from 18222be to 47d3a32 Compare July 29, 2026 18:49
@WatchTree-19

Copy link
Copy Markdown
Contributor Author

Thanks, and apologies for the noise — you're right. The branch had picked up the braille commit from #1990. Rebased onto current main; it's now a single commit touching only packagehallucination.py and its test. The braille change is unaffected (it's still in #1990). Ready for another look.

@jmartin-tech jmartin-tech added detectors work on code that inherits from or manages Detector quality-accuracy This affects result quality/reliability labels Jul 30, 2026
@jmartin-tech
jmartin-tech dismissed their stale review July 31, 2026 14:41

Ready for code review and testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

detectors work on code that inherits from or manages Detector quality-accuracy This affects result quality/reliability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants