Skip to content

Fix self-extension in MultiDict - #1402

Open
cananoo wants to merge 2 commits into
aio-libs:masterfrom
cananoo:fix/1398-self-extension
Open

Fix self-extension in MultiDict#1402
cananoo wants to merge 2 commits into
aio-libs:masterfrom
cananoo:fix/1398-self-extension

Conversation

@cananoo

@cananoo cananoo commented Aug 27, 2026

Copy link
Copy Markdown

What do these changes do?

Make MultiDict.extend safe when the source is the destination itself. The C extension and pure Python implementation snapshot the original items before appending, including proxy aliases.

Are there changes in behavior for the user?

Yes. mapping.extend(mapping) now completes and appends exactly one copy of the original entries in their original order.

Is it a substantial burden for the maintainers to support this?

No. The change is limited to the existing self-alias paths and regression coverage for both implementations and proxy aliases.

Related issue number

Fixes #1398

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes (N/A: the existing API documentation does not describe self-extension semantics)
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt (N/A: this repository has no CONTRIBUTORS.txt)
  • Add a new news fragment into the CHANGES/ folder
Agent run details
  • Linux Python 3.12 build with the default C extension and pure Python implementation: 1662 passed, 7 deselected
  • MULTIDICT_NO_EXTENSIONS=1 self and proxy smoke coverage for MultiDict and CIMultiDict: passed
  • Ruff checks and formatting: passed
  • clang-format check: passed
  • make doc-spelling: the current tree reports eight existing spelling warnings in CHANGES.rst; the new fragment introduced no spelling warning

Drafted with OpenAI Codex (GPT-5); pending human review.

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Aug 27, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 242 untouched benchmarks


Comparing cananoo:fix/1398-self-extension (dc216a8) with master (4c75c93)

Open in CodSpeed

@Vizonex Vizonex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cananoo I like your approach to this but there are some things your going to have to do before you can have my approval on merging this PR.

  • 3.14 in the workflows seems to be broken and needs fixing, if you do not have 3.14 my recommendation is to create a virtual environment using the uv package manager system and run tests/test_multidict.py using pytest.

@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

I followed up on the 3.14 request. The failing 3.14/3.14t jobs do not report a self-extension assertion failure: they error during pytest-cov teardown because coverage's sysmon core emits no-sysmon-context for --cov-context=test on Python 3.14/3.14t. The same coverage failure is present in the repository's scheduled master CI.

On this Windows host I cannot build the C extension because MSVC is unavailable, but I installed the branch in pure-Python mode and ran the requested target: tests/test_multidict.py passed with 432 passed, 430 deselected. The initial Linux C/Python run also passed 1662 tests; I have left the unrelated CI coverage configuration unchanged.

@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

I followed up on the 3.14 request. The failing 3.14/3.14t jobs do not report a self-extension assertion failure: they error during pytest-cov teardown because coverage's sysmon core emits no-sysmon-context for --cov-context=test on Python 3.14/3.14t. The same coverage failure is present in the repository's scheduled master CI.

On this Windows host I cannot build the C extension because MSVC is unavailable, but I installed the branch in pure-Python mode and ran the requested target: tests/test_multidict.py passed with 432 passed, 430 deselected. The initial Linux C/Python run also passed 1662 tests.

I also added a small CI-only follow-up that selects ctrace for CPython and pytrace for PyPy, so Python 3.14/3.14t no longer select the incompatible sysmon core. The workflow YAML parses successfully, and the pure-Python target passes with COVERAGE_CORE=ctrace.

@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.86%. Comparing base (41c1b91) to head (dc216a8).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1402   +/-   ##
=======================================
  Coverage   99.86%   99.86%           
=======================================
  Files          28       28           
  Lines        3627     3639   +12     
  Branches      265      268    +3     
=======================================
+ Hits         3622     3634   +12     
  Misses          3        3           
  Partials        2        2           
Flag Coverage Δ
CI-GHA 99.86% <100.00%> (+<0.01%) ⬆️
pytest 99.86% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

The follow-up CI run is complete. All 3.14 and 3.14t extension/non-extension jobs now pass on Linux, macOS, and Windows, including the debug coverage paths. Codecov reports 99.86% project coverage with 100% patch coverage, and CodSpeed reports no performance change. The only remaining red status is Coveralls' project-level report (85.257%, a 0.03% decrease); its patch coverage is 100%, so this appears to be the existing aggregate threshold rather than an uncovered line in this change. The PR is now ready for review.

@cananoo
cananoo marked this pull request as ready for review August 28, 2026 03:02
@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

@Vizonex The requested 3.14/3.14t CI follow-up is complete: all extension and pure-Python jobs pass across Linux, macOS, and Windows. I also marked the PR ready for review. Would you mind taking another look when convenient?

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge, with both implementations consistently snapshotting self-alias inputs before mutation.

The changed paths preserve entry order, duplicate values, proxy alias behavior, case-insensitive key processing, and temporary-object cleanup, and the parameterized regression test exercises the public self-extension behavior across implementations.

Reviews (1): Last reviewed commit: "Fix coverage on Python 3.14 CI" | Re-trigger Greptile

@cananoo

cananoo commented Aug 28, 2026

Copy link
Copy Markdown
Author

I addressed the Python 3.14 CI issue in dc216a8 by selecting the coverage tracer appropriate for each interpreter. The 3.14/3.14t extension and non-extension jobs now pass across Linux, macOS, and Windows; tests/test_multidict.py passes (432 selected tests), and the patch coverage is 100%. Could you please re-review this PR? @Vizonex

@Vizonex
Vizonex self-requested a review August 28, 2026 16:24

@Vizonex Vizonex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great Job, lets see what the others have to say.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MultiDict.extend segfaults while iterating its own entries

2 participants