Skip to content

test: Collect the test files in one place - #1680

Merged
chfast merged 1 commit into
masterfrom
test/collect-test-files
Aug 29, 2026
Merged

test: Collect the test files in one place#1680
chfast merged 1 commit into
masterfrom
test/collect-test-files

Conversation

@chfast

@chfast chfast commented Aug 28, 2026

Copy link
Copy Markdown
Member

Both drivers walked the test directory themselves, and the two walks were the same code: the
recursive search, the .json filter, the index.json skip, the sort, and the suite name taken
from the file's directory relative to the root. Only the registration at the end differed.
collect_test_files() does it once. --gtest_list_tests over the EEST fixtures stays
byte-identical to master, 20245 names for the state tool and 21333 for the blockchain one, in the
same order.

The integration tests grow to cover what they were missing, each addition checked to raise the
covered-line count on its own: the blockchain tool had only ever been given single files, so the
whole directory form went unrun; SuiteA gains an index.json and a notes.txt, which the
existing tests1_list expectation would show if either were collected; a fixture with a block
shape the loader rejects covers both places that is reported; and -k over a directory is
filtered per case rather than at registration. blockchaintest.cpp goes from 61% to 97% of lines
and 100% of branches, test_files.cpp to 100% throughout.

@codspeed-hq

codspeed-hq Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 129 untouched benchmarks


Comparing test/collect-test-files (4c1edc9) with master (009d950)

Open in CodSpeed

@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.93%. Comparing base (009d950) to head (4c1edc9).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1680      +/-   ##
==========================================
+ Coverage   97.86%   97.93%   +0.06%     
==========================================
  Files         176      178       +2     
  Lines       15995    15995              
  Branches     3666     3664       -2     
==========================================
+ Hits        15654    15664      +10     
+ Misses        255      252       -3     
+ Partials       86       79       -7     
Flag Coverage Δ
eest-develop 87.69% <89.47%> (+0.03%) ⬆️
eest-develop-gmp 26.13% <78.94%> (+0.04%) ⬆️
eest-legacy 16.87% <78.94%> (+0.05%) ⬆️
eest-libsecp256k1 28.35% <78.94%> (+0.04%) ⬆️
eest-stable 87.69% <89.47%> (+0.03%) ⬆️
evmone-unittests 94.14% <100.00%> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.19% <ø> (ø)
tooling 93.43% <100.00%> (+0.53%) ⬆️
tests 99.81% <ø> (ø)
Files with missing lines Coverage Δ
test/blockchaintest/blockchaintest.cpp 96.36% <100.00%> (+7.65%) ⬆️
test/statetest/statetest.cpp 97.14% <100.00%> (+6.11%) ⬆️
test/utils/test_files.cpp 100.00% <100.00%> (ø)
test/utils/test_files.hpp 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chfast
chfast force-pushed the test/collect-test-files branch 9 times, most recently from 1ff008c to 10a57b2 Compare August 29, 2026 10:52
@chfast
chfast requested a balanced review from Copilot August 29, 2026 12:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Centralizes JSON fixture discovery for both test runners and expands integration coverage.

Changes:

  • Adds shared, sorted test-file collection.
  • Updates state and blockchain test registration.
  • Adds filtering, directory, tracing, and unsupported-fixture coverage.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/utils/test_files.hpp Declares shared test discovery API.
test/utils/test_files.cpp Implements filtered, sorted recursive discovery.
test/utils/CMakeLists.txt Builds the new utility.
test/statetest/statetest.cpp Uses shared discovery.
test/blockchaintest/blockchaintest.cpp Uses shared discovery.
test/integration/statetest/CMakeLists.txt Expands integration coverage.
test/integration/statetest/tests1/SuiteA/index.json Tests index exclusion.
test/integration/statetest/tests1/SuiteA/notes.txt Tests non-JSON exclusion.
test/integration/blockchaintest/CMakeLists.txt Adds directory and error-path tests.
test/integration/blockchaintest/unsupported_rlp.json Exercises unsupported block fixtures.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread test/integration/statetest/CMakeLists.txt Outdated
Both drivers walked the directory themselves, with the same recursive
search, the same index.json skip, the same sort and the same suite naming
written out twice. Only the registration at the end differed.

collect_test_files() in testutils does the walk once and returns each file
with the suite it belongs to. The listed tests are unchanged, name for
name: 20245 for the state tool and 21333 for the blockchain one, in the
same order.

The integration tests grow to cover the paths they were missing. The
blockchain tool was only ever given single files, so the whole directory
form went unrun; SuiteA gains files that must not be collected; and a
fixture with an unsupported block shape exercises both places that shape is
reported. Coverage of blockchaintest.cpp goes from 61% to 97% of lines, and
of the collector to 100%.
@chfast
chfast force-pushed the test/collect-test-files branch from 10a57b2 to 4c1edc9 Compare August 29, 2026 14:44
@chfast
chfast merged commit 2efd762 into master Aug 29, 2026
25 checks passed
@chfast
chfast deleted the test/collect-test-files branch August 29, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants