Skip to content

Yul Optimizer: Avoid full reference scan in DataFlowAnalyzer::clearValues - #16925

Open
0xferit wants to merge 2 commits into
argotorg:developfrom
0xferit:perf/dataflowanalyzer-clearvalues-reverse-index
Open

Yul Optimizer: Avoid full reference scan in DataFlowAnalyzer::clearValues#16925
0xferit wants to merge 2 commits into
argotorg:developfrom
0xferit:perf/dataflowanalyzer-clearvalues-reverse-index

Conversation

@0xferit

@0xferit 0xferit commented Aug 18, 2026

Copy link
Copy Markdown

Description

Large codebases pay double-digit minutes per --via-ir build (#13050, #15141); on pathological shapes this scan dominates (#13822). DataFlowAnalyzer::clearValues() has to find the variables whose currently assigned expression references one being cleared; it did so by scanning all of m_state.sortedReferences, a walk over every tracked variable per call. This maintains the inverse relation (reverseReferences), so referencers are looked up directly.

Both directions are written only by two private helpers; removing a forward edge asserts that exactly one matching reverse edge is removed. hasNonemptyIntersectionSorted() served only the scan and goes with it.

argotorg:develop this PR measured
Basic operation one YulName key inspection: a comparison in the sorted scan one YulName key inspection: a hash probe constants differ; priced below
Reference-bookkeeping time, cumulative worst case O(n³) expected O(n²) -34.0% on chains.sol --via-ir, +2.4% on verifier.sol
Asymptotic auxiliary space O(n²) O(n²), one reverse edge per forward edge peak resident set size (RSS) +0.4% to 1.0%

n is the abstract syntax tree node count of one function body. The cubic requires handleAssignment storing one wide reference vector for many assigned names; typical code sits far from it. Only chains.sol --via-ir below resolves; its shape, long chains of narrow assignments, is cumulatively quadratic before, expected linear after.

configuration argotorg:develop this PR
chains.sol --optimize --via-ir 1766.7 ms 1166.2 ms -34.0%
chains.sol --optimize 48.2 ms 48.4 ms within noise
OptimizorClub.sol --optimize --via-ir 583.7 ms 590.4 ms within noise
verifier.sol --optimize --via-ir 117.1 ms 119.9 ms +2.4%

Medians of 22 counterbalanced paired observations per cell, Release builds, macOS arm64; laptop numbers, a dedicated-machine run is welcome. The last row prices maintaining the index: head slower in all 22 pairs. Timings compare parent f5395d3bc against d8acc5ad8, whose DataFlowAnalyzer.cpp is byte-identical to head.

No intended change to compiler output: bytecode_size is identical in all 58 project/preset cells of the c_ext_benchmarks artifacts (head against parent), and local metadata-stripped (--no-cbor-metadata) bytecode is byte-identical in all four configurations. The gas columns move for unrelated reasons: deployment-gas deltas are mostly exact multiples of 12, the zero/nonzero calldata cost gap from the commit-specific metadata hash, and method-gas deltas appear even under legacy-no-optimize, where the Yul optimizer never runs; the comparison is mine.

No new test: the suite detects both directions of the relation breaking. Omitting the reverse-edge insertion fails 221 of 651 yulOptimizerTests; omitting its removal fails exactly commonSubexpressionEliminator/clear_not_needed.

Prior art. Same approach as #14112, closed as inconclusive rather than rejected; #13822 carries no desirability label and may want triage first. @cameel's open question there, whether one pathological contract justifies the cost, stands. New since then: #14112's 23.16 s to 10.52 s reproduces on a second toolchain (1766.7 to 1166.2 ms here); the regression cost is measured (+2.4%, RSS +0.4% to 1.0%); the diff is narrower (+70/-37 against +135/-8).

Reproduction: raw benchmark data gist with build commands (raw.txt), timing driver (bench.py), mutation commands (mutations.md), all timings, RSS readings and output hashes.

Checklist

AI Disclosure

Claude Code: profiling, implementation, benchmarks, test runs, and the first draft of this description. OpenAI Codex: independent review of the implementation, the benchmark claims and the reproduction commands, which is where several corrections above came from. I reviewed, understood and verified all of it myself.

@0xferit
0xferit marked this pull request as draft August 18, 2026 16:48
@0xferit
0xferit force-pushed the perf/dataflowanalyzer-clearvalues-reverse-index branch 6 times, most recently from 88b2b18 to cc613f0 Compare August 18, 2026 23:21
@0xferit
0xferit marked this pull request as ready for review August 19, 2026 00:39
@0xferit
0xferit marked this pull request as draft August 19, 2026 01:59
@0xferit
0xferit force-pushed the perf/dataflowanalyzer-clearvalues-reverse-index branch 5 times, most recently from f639842 to 364454c Compare August 19, 2026 11:39
…lues

clearValues() scanned every entry of m_state.sortedReferences on each call
to find the variables whose last assigned expression references one of the
variables being cleared. Maintain the inverse relation instead, so that
those referencers are looked up directly:

  reverseReferences[b] = {a | b in sortedReferences[a]}

Both directions are written only by two private helpers, and removing a
forward edge asserts that exactly one matching reverse edge is removed.

hasNonemptyIntersectionSorted() was introduced for the removed scan and has
no other caller, so it goes away with it.

No intended change to metadata-independent compiler output: bytecode
compiled with --no-cbor-metadata is byte-identical on the repository
benchmarks, and bytecode size is unchanged in all 58 project/preset cells
that the external benchmark job reports.
@0xferit
0xferit force-pushed the perf/dataflowanalyzer-clearvalues-reverse-index branch from 364454c to 30cac9f Compare August 19, 2026 12:24
@0xferit 0xferit changed the title Yul Optimizer: Avoid quadratic scan in DataFlowAnalyzer::clearValues Yul Optimizer: Avoid full reference scan in DataFlowAnalyzer::clearValues Aug 19, 2026
@0xferit
0xferit marked this pull request as ready for review August 19, 2026 17:00
@0xferit

0xferit commented Aug 19, 2026

Copy link
Copy Markdown
Author

@blishko Following your note on #16926: this is the one, now marked ready for review. #16926 stays in draft; the diffs are independent and non-overlapping, so it can wait its turn.

@blishko

blishko commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

@0xferit, thanks for the contribution! There are currently several things that have higher priority so it may take some time to analyze this PR properly. Apologies for that, but we have to prioritize at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants