Yul Optimizer: Avoid full reference scan in DataFlowAnalyzer::clearValues - #16925
Open
0xferit wants to merge 2 commits into
Open
Yul Optimizer: Avoid full reference scan in DataFlowAnalyzer::clearValues#169250xferit wants to merge 2 commits into
0xferit wants to merge 2 commits into
Conversation
0xferit
marked this pull request as draft
August 18, 2026 16:48
2 tasks
0xferit
force-pushed
the
perf/dataflowanalyzer-clearvalues-reverse-index
branch
6 times, most recently
from
August 18, 2026 23:21
88b2b18 to
cc613f0
Compare
0xferit
marked this pull request as ready for review
August 19, 2026 00:39
0xferit
marked this pull request as draft
August 19, 2026 01:59
0xferit
force-pushed
the
perf/dataflowanalyzer-clearvalues-reverse-index
branch
5 times, most recently
from
August 19, 2026 11:39
f639842 to
364454c
Compare
…lues
clearValues() scanned every entry of m_state.sortedReferences on each call
to find the variables whose last assigned expression references one of the
variables being cleared. Maintain the inverse relation instead, so that
those referencers are looked up directly:
reverseReferences[b] = {a | b in sortedReferences[a]}
Both directions are written only by two private helpers, and removing a
forward edge asserts that exactly one matching reverse edge is removed.
hasNonemptyIntersectionSorted() was introduced for the removed scan and has
no other caller, so it goes away with it.
No intended change to metadata-independent compiler output: bytecode
compiled with --no-cbor-metadata is byte-identical on the repository
benchmarks, and bytecode size is unchanged in all 58 project/preset cells
that the external benchmark job reports.
0xferit
force-pushed
the
perf/dataflowanalyzer-clearvalues-reverse-index
branch
from
August 19, 2026 12:24
364454c to
30cac9f
Compare
0xferit
marked this pull request as ready for review
August 19, 2026 17:00
Author
Contributor
|
@0xferit, thanks for the contribution! There are currently several things that have higher priority so it may take some time to analyze this PR properly. Apologies for that, but we have to prioritize at the moment. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Large codebases pay double-digit minutes per
--via-irbuild (#13050, #15141); on pathological shapes this scan dominates (#13822).DataFlowAnalyzer::clearValues()has to find the variables whose currently assigned expression references one being cleared; it did so by scanning all ofm_state.sortedReferences, a walk over every tracked variable per call. This maintains the inverse relation (reverseReferences), so referencers are looked up directly.Both directions are written only by two private helpers; removing a forward edge asserts that exactly one matching reverse edge is removed.
hasNonemptyIntersectionSorted()served only the scan and goes with it.YulNamekey inspection: a comparison in the sorted scanYulNamekey inspection: a hash probeO(n³)O(n²)chains.sol --via-ir, +2.4% onverifier.solO(n²)O(n²), one reverse edge per forward edgenis the abstract syntax tree node count of one function body. The cubic requireshandleAssignmentstoring one wide reference vector for many assigned names; typical code sits far from it. Onlychains.sol --via-irbelow resolves; its shape, long chains of narrow assignments, is cumulatively quadratic before, expected linear after.chains.sol --optimize --via-irchains.sol --optimizeOptimizorClub.sol --optimize --via-irverifier.sol --optimize --via-irMedians of 22 counterbalanced paired observations per cell, Release builds, macOS arm64; laptop numbers, a dedicated-machine run is welcome. The last row prices maintaining the index: head slower in all 22 pairs. Timings compare parent
f5395d3bcagainstd8acc5ad8, whoseDataFlowAnalyzer.cppis byte-identical to head.No intended change to compiler output:
bytecode_sizeis identical in all 58 project/preset cells of thec_ext_benchmarksartifacts (head against parent), and local metadata-stripped (--no-cbor-metadata) bytecode is byte-identical in all four configurations. The gas columns move for unrelated reasons: deployment-gas deltas are mostly exact multiples of 12, the zero/nonzero calldata cost gap from the commit-specific metadata hash, and method-gas deltas appear even underlegacy-no-optimize, where the Yul optimizer never runs; the comparison is mine.No new test: the suite detects both directions of the relation breaking. Omitting the reverse-edge insertion fails 221 of 651
yulOptimizerTests; omitting its removal fails exactlycommonSubexpressionEliminator/clear_not_needed.Prior art. Same approach as #14112, closed as inconclusive rather than rejected; #13822 carries no desirability label and may want triage first. @cameel's open question there, whether one pathological contract justifies the cost, stands. New since then: #14112's 23.16 s to 10.52 s reproduces on a second toolchain (1766.7 to 1166.2 ms here); the regression cost is measured (+2.4%, RSS +0.4% to 1.0%); the diff is narrower (+70/-37 against +135/-8).
Reproduction: raw benchmark data gist with build commands (
raw.txt), timing driver (bench.py), mutation commands (mutations.md), all timings, RSS readings and output hashes.Checklist
AI Disclosure
Claude Code: profiling, implementation, benchmarks, test runs, and the first draft of this description. OpenAI Codex: independent review of the implementation, the benchmark claims and the reproduction commands, which is where several corrections above came from. I reviewed, understood and verified all of it myself.