Skip to content

Record the 129x129 Ghia cavity results and report steps run per backend - #212

Open
shaia wants to merge 3 commits into
masterfrom
ghia-129-results
Open

shaia wants to merge 3 commits into
masterfrom
ghia-129-results

Conversation

@shaia

@shaia shaia commented Sep 14, 2026

Copy link
Copy Markdown
Owner

The 129×129 Ghia results table in docs/validation/cavity-backends-validation.md still said TBD, and ROADMAP §6.1 listed it as the main open P0 item for v1.0. No new run was needed: since the Re=400/1000 cases landed, the EC2 workflow (gpu-validation.yml) has built with -DCAVITY_FULL_VALIDATION=ON and run every CavityBackend_* case on each push to master. This PR records those results and closes the Ghia item. It also covers two gaps found along the way: the Explicit Euler cases stop early (recorded, not fixed), and scripts/ec2-validate.sh never ran Re=400/1000 (fixed).

Results

Three EC2 runs gave identical values: 34853615452 (6e1b104), 34815324268 (23b196a) and 30116961115 (#201). AVX2, OpenMP and CUDA agree to four decimals:

Re RMS_u RMS_v Step budget × dt (t)
100 0.0017 0.0024 50000 × 0.0002 (10)
400 0.0096 0.0328 60000 × 0.0005 (30)
1000 0.0299 0.0300 100000 × 0.0005 (50)

Every case is at least 3× under the 0.10 target, so the step budgets stay unchanged.

What changed

Test output

  • test_backend_validation() now prints Steps run: N/budget stopped by KE threshold: yes|no KE residual: X for each backend. It only adds output; pass/fail logic is unchanged.
  • "Stopped by KE threshold" means the per-step kinetic-energy change fell below 1e-8. It is not a convergence claim: the Euler cases below hit it with an undeveloped flow.
  • The line avoids RMS_u:, SKIPPED and [FAILED], the strings the workflow's summary awk keys on. It is named Steps run: so it can't be confused with the Steps: header that main() already prints.

Explicit Euler cases stop early (recorded, not fixed)

  • The new line shows the Euler cases ending through the harness's kinetic-energy exit (relative change per step below 1e-8). Their solvers cap dt at 1e-4, so the runs reach only t ≈ 1.1–1.2:
    • 129×129: 11,323 (AVX2) and 11,775 (OpenMP) of 250,000 steps
    • 33×33 CI: 11,300 of 25,000 steps (scalar)
  • That is why their RMS_u (0.0957) is the same on both grids. The flow needs about 10–20 time units to reach steady state, so these runs pass the 0.15 target without a developed solution.
  • Recorded as a Known Issue in the validation doc and as a new open item in ROADMAP §6.1.

scripts/ec2-validate.sh

  • It ran test_cavity_backends with no filter, which skips the Re=400/1000 cases because they need an explicit filter. It now runs ctest -R '^CavityBackend_' --no-tests=error, which covers exactly the 12 entries registered in CMakeLists.txt.
  • Under set -euo pipefail, a failing test binary ended the script before EXIT_CODE=$? ran, so the FAILED banner never printed. The status is now captured with || EXIT_CODE=$?.
  • It schedules tests like CI: ctest -j 4, with OMP_NUM_THREADS = nproc / 4 unless --threads is given.
  • Output is teed to validation_output.txt, and the backend, steps and RMS lines are printed as a summary at the end.
  • The quick-start now uses --release, matching CI.

Docs

  • docs/validation/cavity-backends-validation.md:
    • results table with run provenance;
    • Known Issue section;
    • the CI/CD section now describes the EC2 workflow;
    • stale projection_jacobi_gpu renamed to projection_gpu;
    • Next Steps rewritten.
  • docs/validation/lid-driven-cavity.md:
    • full-validation settings (previously "3000–10000 steps" and a #define);
    • a 129×129 results summary;
    • the 33×33 CI section and Fast Mode step count (previously ~0.10 / ~0.08 RMS, 1500–4000 steps and "reaches steady state"). It now gives the measured 5000-step run: RMS 0.0382 / 0.0440, still far from steady state;
    • the pressure-solver list (previously "Red-Black SOR… GPU: Jacobi").
  • docs/reference/solvers.md: measured RMS replaces "RMS error < 0.01", and the broken validation/… link is fixed.
  • ROADMAP.md §6.1:
    • Ghia item ticked;
    • section note and CI vs Release table updated;
    • Euler follow-up added.
    • Cross-architecture consistency stays open: CavityBackend_Consistency compares CPU, AVX2 and OpenMP at 33×33 and never includes the GPU.
  • CHANGELOG.md: Added and Fixed entries.

Verification

Windows, MSVC with OpenMP.

  • Full-validation build (Release, AVX2 on, CUDA off, -DCAVITY_FULL_VALIDATION=ON):
    • euler_avx2, euler_omp and consistency pass and print the new line. Euler RMS matches EC2 exactly (0.0957 / 0.1293 and 0.0957 / 0.1277).
    • ctest -N -R "^CavityBackend_" lists exactly 12 tests.
    • The summary awk from gpu-validation.yml, run on that output, still produces correct rows.
  • CI-mode build (Debug):
    • projection_scalar runs 5000/5000 steps with RMS 0.0382 / 0.0440, as documented.
    • euler_scalar passes and prints stopped by KE threshold: yes.
  • bash -n scripts/ec2-validate.sh passes, and the new relative doc links resolve.
  • Not verified:
    • the projection cases locally (their numbers come from EC2) and the GPU backend;
    • ec2-validate.sh end to end on Linux.
  • Still to come: the Steps run: lines for the projection cases, in the first EC2 run after merge (or earlier if this PR is labelled gpu-test).

🤖 Generated with Claude Code

The EC2 validation workflow has run the 129x129 cavity cases on every master
push since June, but the results table still said TBD. Runs 34853615452,
34815324268 and 30116961115 gave identical RMS for the AVX2, OpenMP and CUDA
projection backends at Re=100/400/1000, all at least 3x under the 0.10
target, so the step budgets stay and the ROADMAP 6.1 Ghia item is done.
Cross-architecture consistency stays open: its 0.1% check never includes
the GPU.

Each backend line now prints the steps it ran. That shows the Explicit Euler
cases stopping near 11,300 steps through the kinetic-energy exit on both the
33x33 and 129x129 grids, which is why their RMS does not change with grid
size; the validation doc and ROADMAP record it as a follow-up.

ec2-validate.sh ran the test binary without a filter, which skips Re=400 and
Re=1000, and set -e swallowed its failure banner. It now runs the registered
CavityBackend_ ctest entries and reports the result.
@shaia shaia self-assigned this Sep 15, 2026
@shaia
shaia requested a balanced review from Copilot September 15, 2026 02:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new output mislabels a KE threshold as convergence, and the updated 33×33 section retains stale results and settings.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Records validated 129×129 Ghia cavity results and improves EC2 validation reporting.

Changes:

  • Documents backend results, provenance, and the Explicit Euler limitation.
  • Reports actual step counts and KE stopping status.
  • Runs all CavityBackend_* tests with improved logging and failure handling.
File summaries
File Description
tests/validation/test_cavity_backends.c Adds step and KE-residual output.
scripts/ec2-validate.sh Runs all backend cases and summarizes results.
ROADMAP.md Records validation completion and Euler follow-up.
docs/validation/lid-driven-cavity.md Updates validation configuration and results.
docs/validation/cavity-backends-validation.md Adds full results, provenance, and known issue.
docs/reference/solvers.md Updates measured RMS results and links.
CHANGELOG.md Records validation and script changes.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/validation/test_cavity_backends.c Outdated
Comment thread docs/validation/lid-driven-cavity.md Outdated
@shaia
shaia requested a balanced review from Copilot September 15, 2026 05:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Renaming the heading to 33x33 made the old ~0.10/~0.08 RMS, 3000-4000 steps
and "reaches steady state" read as current CI behavior (review on #212). The
CI run is 5000 steps at dt 0.0005 with RMS 0.0382/0.0440, and it ends with a
per-step kinetic-energy change of 2.7e-5, far from steady state. The Fast
Mode step count above it was stale the same way.
result.converged only means the per-step kinetic-energy change fell below
1e-8. The Explicit Euler cases report it at t~1.1 with an undeveloped flow,
so "converged: yes" read as a physical conclusion (review on #212).
@shaia
shaia requested a balanced review from Copilot September 15, 2026 13:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants