Replace Boost.spirit - #401
Draft
marcosbento wants to merge 7 commits into
Draft
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #401 +/- ##
===========================================
- Coverage 52.24% 52.13% -0.11%
===========================================
Files 1251 1258 +7
Lines 102109 103043 +934
Branches 15183 15382 +199
===========================================
+ Hits 53347 53723 +376
- Misses 48762 49320 +558 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
marcosbento
force-pushed
the
task/replace_boost_spirit
branch
2 times, most recently
from
July 28, 2026 07:40
4ce1f82 to
cd17b90
Compare
marcosbento
marked this pull request as draft
July 28, 2026 08:12
Stand up the scaffold for the Boost.Spirit -> handcrafted parser migration. The new Boost.Test target u_expr_v1_v2 (libs/node/ CMakeLists.txt) exercises the expression parser against: - a compact canonical set covering every operand type and operator form (taken from the existing TestExprParser.cpp list), and - the full golden corpus (corpus.json, 2,156+ real-world expressions). Previously, the harness validates V1 against itself: each expression is parsed twice with ExprDuplicate flushed between runs, confirming that corpus loading, cache-flush wiring, and ParseResult capture all work correctly. The same infra will be extended in Phase 2 to compare V1 and V2 side-by-side. Three test cases: test_v1_canonical_consistency - canonical list, V1 deterministic test_v1_corpus_valid_consistency - all 2156 valid corpus exprs test_v1_corpus_invalid_rejection - all invalid corpus exprs rejected Re ECFLOW-1922
Relocate the Boost.Spirit implementation into namespace ecf::expression::v1, making the facade header the sole consumer-facing interface: - ExprParser.hpp becomes a thin facade: the two using-aliases (ExprParser, SimpleExprParser) point to versioned namespace ecf::expression::v1. To cut over to V2, only those two lines change. - ExprParserV1.hpp declares ecf::expression::v1::ExprParser with full Doxygen documentation. - ExprParserV1.cpp is the full Boost.Spirit implementation, wrapped in namespace ecf::expression::v1. - SimpleExprParser.cpp/hpp declares ecf::expression::SimpleExprParser, with full Doxygen documentation. - libs/CMakeLists.txt updated: ExprParser.cpp -> ExprParserV1.cpp in the source list; ExprParserV1.hpp added to the header list. Zero behaviour change verified by tests. Re ECFLOW-1922
ExprParserV2.hpp/.cpp introduce namespace ecf::expression::v2::ExprParser with the same public API as V1 but returning a 'not yet implemented' failure in doParse(). Both TUs are always compiled (libs/CMakeLists.txt updated) so the differential harness exercises both implementations. The u_expr_v1_v2 harness gains three new test cases: test_v1_v2_canonical_differential - compares V1/V2 on the canonical set test_v1_v2_corpus_valid_differential - compares V1/V2 on all valid corpus exprs test_v1_v2_corpus_invalid_differential - V1/V2 must agree on rejections All three are correctly red (V2 stub rejects every expression with an explicit diagnostic). The V1 consistency and corpus tests remain green. Re ECFLOW-1922
Replace the Phase 2 V2 stub with a hand-written lexer and recursive-descent parser for trigger and complete expressions. V2 builds the existing Ast hierarchy, preserving V1 semantics: - greedy word-boundary tokenisation and contiguous path:attribute forms; - node/event states, flags, calendar functions, datetime values, paths and variables; - equal-precedence left-associative arithmetic plus V1's asymmetric boolean associativity; - exact not-spelling rendering and ExprDuplicate cache behaviour. The parser implements save/restore token-stream state to mirror Spirit's ordered alternatives around arithmetic and boolean parentheses. It also preserves observable V1 compatibility quirks documented in the source, including numeric and keyword-shaped node names, repeated path separators, context-sensitive slash lexing, and legacy malformed corpus recoveries. Re ECFLOW-1922
Add an extractor that records raw trigger and complete expression parts from checkpoint files in source order, retaining duplicates for cache-realistic benchmarking. Add p_expr_v1_v2 to validate V1/V2 AST equivalence outside timed sections and measure each implementation separately on full cache-active and distinct cache-cold workloads. Each pass starts from an empty ExprDuplicate cache, so V1 and V2 never share warmed entries. Considering 98 checkpoint datasets contain 4,125,637 raw parts. V2 outperforms V1 in both modes on the meaningful real workloads, so the V1 SimpleExprParser fast path remains V1-only. Real-data preflight also extends V2 compatibility for numeric-leading paths, high-bit identifier bytes, and contextual numeric node names. Re ECFLOW-1922
Point the public ExprParser facade at ecf::expression::v2::ExprParser. Complete V1 compatibility discovered by running the full node suite through the production alias: root flag references, numeric-leading attributes, malformed relative path rejection, invalid datetime diagnostics, chained not operands, and valid flag-name validation. V1 remains compiled for the differential and performance suites, making the cutover reversible while V2 is exercised in normal production entry points. Re ECFLOW-1922
Move test/v1_v2/{TestExprParserV1V2,ExprParserV1V2Benchmark}* to
test/expressions/, renaming the ecbuild targets (u_expr_v1_v2 ->
u_expressions_parser, p_expr_v1_v2 -> p_expressions_parser_benchmark)
since the suite no longer only compares V1 against V2.
Point both performance tests at the relocated checkpoint-expression
JSON corpus, skipping datasets with fewer than 10 distinct
expressions so benchmark output stays meaningful.
Add extract_expressions.py, used to (re)generate the per-checkpoint
expression JSON corpus from raw ecFlow checkpoint files, and
DefsCheckBenchmark.cpp / p_defs_check_benchmark, measuring
Defs::check() wall time (cold vs warm ExprDuplicate cache) for
whichever parser the library was built against.
Re ECFLOW-1922
marcosbento
force-pushed
the
task/replace_boost_spirit
branch
from
July 28, 2026 17:13
cd17b90 to
c4b3490
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
As per PR title.
Contributor Declaration
By opening this pull request, I affirm the following:
🌦️ >> Documentation << 🌦️
https://sites.ecmwf.int/docs/dev-section/ecflow/pull-requests/PR-401