Experimental performance family (default-off): load-balance infrastructure, active-box windowing, block-structured AMR, hybrid WENO/Riemann sensors#1628
Open
sbryngelson wants to merge 312 commits into
Conversation
…tate-based reflux apply (inert)
…, time-interpolated ghosts, accumulated reflux
…validated with fine-level advance)
…ex sweep, owner-guarded fine machinery
…lamp, owner-only rebuild
…havior-identical)
…bracketed coupling (M1); lift GPU prohibit
…p, golden coverage
…sters, on-device ghost fill/RK/restriction
…fine halo exchange, distributed flux registers
…anning patches move freely; drop owner-window model
…alance coupling, deterministic feasibility clamp, fine-advance rank timing
# Conflicts: # src/common/m_mpi_common.fpp
…tes + 3D golden case
…m-preserving alpha prolongation, mpp_lim-gated consistency
…ersistence (SP10)
…ay); num_patches=1 behavior-identical
…ion merge, per-slot advance (SP12a)
…sambiguate from IC patches; golden values unchanged
…moment prolongation, per-block bubble state
…ation on the fine level
… (fm host array referenced in GPU_PARALLEL_LOOP bounds) s_amr_fine_slice (the c8b1006 device-packed seam halo) used the host local array fm(TB)/fm(TA) as GPU_PARALLEL_LOOP bounds, so nvfortran/Cray demanded fm PRESENT on device -> runtime 'FATAL ERROR: data in PRESENT clause was not found on device: name=fm(2:)'. AMD flang tolerated it; NVHPC (Phoenix) and CCE (Frontier) gpu-acc lanes aborted every np>=2 AMR test. Use the scalars na-1/nb-1 (na=fm(TA)+1 already computed on host) as the bounds - byte-identical, no host-array reference in the device region. Validated: 6 np=2 AMR goldens (244B1E42 F57C3A5B ADA042A2 EF58E377 4644A339 B7704247) pass on 2x H200 gpu-acc.
… (fm host array referenced in GPU_PARALLEL_LOOP bounds) s_amr_fine_slice (the c8b1006 device-packed seam halo) used the host local array fm(TB)/fm(TA) as GPU_PARALLEL_LOOP bounds, so nvfortran/Cray demanded fm PRESENT on device -> runtime 'FATAL ERROR: data in PRESENT clause was not found on device: name=fm(2:)'. AMD flang tolerated it; NVHPC (Phoenix) and CCE (Frontier) gpu-acc lanes aborted every np>=2 AMR test. Use the scalars na-1/nb-1 (na=fm(TA)+1 already computed on host) as the bounds - byte-identical, no host-array reference in the device region. Validated: 6 np=2 AMR goldens (244B1E42 F57C3A5B ADA042A2 EF58E377 4644A339 B7704247) pass on 2x H200 gpu-acc.
…d) dense tag field (per-rank memory now O(#tagged), byte-identical boxes)
…~1290^3 (2048^3+ target) The sparse-clustering encode gi+(mg+1)*(gj+(ng+1)*gk) in default integer overflows once (m_glb+1)*(n_glb+1)*(p_glb+1) >= 2^31 (~1290^3, and the 2048^3 this scaling work targets), silently corrupting decoded box coordinates. Widen locidx/allidx to integer(8) with int8 encode/decode and gather via MPI_INTEGER8; counts/displacements stay int32 (they count tagged cells, < 2^31). Byte-identical at tested grids (7 regrid/multilevel goldens pass).
…ustering compute O(nboxes*ntag) -> O(ntag*depth); byte-identical boxes)
Bumps the ffmt formatter pin in toolchain/pyproject.toml from 0.4.3 to 0.4.4 (fixes single-line-construct indent leak, sbryngelson/ffmt#5). Verified as a 0-change no-op on master formatting; Formatting/Lint Toolchain/Python 3.9-3.14 CI all green.
ffmt 0.4.4 (pinned by the prior commit, fixes sbryngelson/ffmt#5) corrects the pre-existing +4 over-indentation of s_amr_cluster/s_amr_regrid (a 0.3.x single-line 'do ...; end do' indent-leak that 0.4.3 tolerated as a buggy fixed point). Whitespace + doc-comment re-wrap only, zero logic change (git diff -w is comment-rewraps); m_amr.fpp and m_sfc_partition.fpp are the only two files affected. Unblocks clean formatting of subsequent edits.
…r level (was per parent-box; byte-identical)
…slice-copy The hybrid-WENO use_central branch set the central (linear) weights with a runtime-bounded whole-array slice assignment, omega(0:weno_num_stencils) = d_cbL/R(:,SV). On gfx90a OpenMP offload, amdflang lowers that array-slice copy through the Fortran runtime _FortranAAssign path, which spills the WENO kernel to scratch (17.9 KB/work-item on AFAR 23.1.0; AGPRs pinned, occupancy collapse) and runs it ~10x slower -- the register/scratch-spill class tracked in ROCm/llvm-project#2909 (upstream llvm/llvm-project#203890). hybrid_weno is a runtime parameter and AMD builds are non-case-optimized, so the branch is compiled even when hybrid_weno is off (the default). Every WENO case on the Frontier gpu-omp (amdflang 23.1.0) benchmark therefore paid the spill, appearing as a ~3-4x per-step slowdown vs master (which lacks the branch). Reconstruct directly from the central weights (weno3/5), or fill omega with an explicit scalar-store loop (weno7), keeping the copy off the _FortranAAssign path. Numerically identical: the non-hybrid else path is unchanged and the central path is algebraically the same. On AFAR 23.1.0 this drops WENO-kernel scratch 17920 B -> 92 B and restores PR per-step to master's level (0.37 -> 0.28 s). Cray, nvfortran, and newer amdflang (23.2.x) were unaffected. Refs: ROCm/llvm-project#2909
…s (serial path) The serial AMR restart reader assumed every block's fine extent was ref_ratio*(region_width)-1 (single level), so it rejected every level>=2 block as 'block fine extents disagree with the region (corrupt file)' -- multi-level (amr_max_level>=2) runs could not restart. A level-l block covers ref_ratio**l fine cells per L0 cell of its region, not ref_ratio (see the work-weight at m_amr.fpp:1173 and s_set_amr_fine_geometry:1297-1321). The parallel_io read path also sizes each block from the region before extents are read, so the level cannot be back-derived -- it must be stored. Fix (serial path): the writer stores amr_block_level(k) in each per-block header; the reader reads it, sets amr_block_level(k) before the owner/geometry rebuild (s_amr_assign_block_owners and s_set_amr_fine_geometry key off it to place L>=2 blocks under their parent), and generalizes the extent check to ref_ratio**level. New test AMR -> 1D -> multi-level restart (4AF96C49) restart_checks the static L2 hierarchy; validated on gpu-acc np=1 (fix passes; single-level AMR restart 21C71558, 1D restart 1A379909, multi-level advance 75AD6885 unchanged). parallel_io path left unchanged (still fails-closed on size mismatch, no regression) -- a clean second increment.
# Conflicts: # src/simulation/m_amr.fpp
… MPI code (PR #7 review C1+I1) C1 (Critical): the parallel_io (MPI-IO) AMR restart path stored no per-block level and hard-coded a level-1 fine extent (ref_ratio*width), so multi-level (amr_max_level>1) restart read every L>=2 block at the wrong resolution and mislaid all downstream block offsets -- caught only by a fragile total-size tripwire. Mirror the serial fix onto the MPI-IO path: the per-block header is now 7 ints (region box + refinement level), the reader sets amr_block_level(k) before the owner/geometry rebuild and sizes each block with ref_ratio**level, with a level-range guard. New restart_check test 'AMR -> 1D -> multi-level restart parallel_io np=2' (78314D65). Single-level MPI-IO restart (5EFB3277) and serial multi-level restart (4AF96C49) unchanged; validated np=2 gpu-acc. I1 (Important): delete dead code. s_mpi_sendrecv_amr_fine_halo (~150 lines + GPU pack/unpack kernels) had zero call sites under the whole-block ownership model that superseded the continuation-face halo, yet s_initialize_amr_mpi_buffers allocated its device buffers (amr_buff_send/recv) on every AMR run. Removed both routines, the buffers, the finalize dealloc, and the call site (max_f1/2/3 stay -- used for slot coordinate sizing); fixed the stale comment referencing the nonexistent s_mpi_bcast_amr_reflux_faces. Net -159 lines.
review Perf-2) s_amr_fine_fine_halo rescanned all O(nblocks^2) block pairs with f_amr_seam every RK stage (6x per fine step) to find same-level adjacent seams. Block topology (regions/levels/count) changes only at regrid/restart, so cache the (xb, yb, seam-dim) list once via s_amr_build_seam_pairs and iterate O(#seams) per stage. Rebuilt when amr_seam_pairs_dirty is set (regrid, restart, init) with a block-count tripwire as backup. The list preserves the original (xb, yb) nested-loop order on all ranks (replicated region metadata), so the paired MPI_SENDRECVs stay matched -> byte-identical. New pure helper f_amr_seam_dim shares the adjacency logic between builder and halo. Validated: full AMR suite 57/57 byte-identical, np=2 gpu-acc.
…bug print (PR #7 review)
…e load_balance debug behind MFC_DEBUG (PR #7 review)
…ratio + np-flexible restart (PR #7 review)
…mirroring m_checker (PR #7 review)
…iew surface reduction)
…l) + parent-coarse invariants; disambiguate 2*r0 (PR #7 review)
…el case.md subsection; move fine-extent/parent-coarse invariants into common-pitfalls; refresh amr_golden_tests docstring (PR #7 review)
…morton/f_morton) (PR #7 review)
…ga/amr-scaling Resolves 11 textual conflicts (bubble sigs=MFlowCode#1648 replay, m_ibm=AMR global ghost_points + MFlowCode#1290 glb_bounds, rest additive) plus 2 semantic conflicts git auto-merged silently: AMR's s_lag_cloud_bbox_local nBubs->n_el_bubs_loc (MFlowCode#1290 moved the local bubble count to m_global_parameters) and s_lag_phys_to_cells/s_amr_body_bbox x/y/z_domain->glb_bounds (MFlowCode#1290 removed x/y/z_domain from sim, replaced w/ grid-derived glb_bounds). CPU-validated: 50/50 pure-AMR + 26/26 non-AMR Lagrange byte-identical; 2 AMR+Lag goldens (4B08E9B7,BCE1BBAE) regenerated for MFlowCode#1290's new EL physics. GPU np>=2 pending.
s_write_amr_restart emits a 7-integer per-block header (region box[6] +
amr_block_level[1]), but the post-process reader s_read_amr_data assumed
a 6-int header and never consumed the level field, shifting every
following field by one integer:
- parallel_io: wext (per-rank fine extents) was read at 6*ibytes
instead of 7*ibytes, so fm picked up level(=1) as the x-extent and a
24^3 fine block decoded as 2x24x24; the conservative-data offset
(ddisp) was one int short, misaligning the payload.
- serial: read(2) reg, rm, rn, rp put the level field into rm.
The misread is deterministic, but the resulting garbage only lands on a
NaN bit pattern on some targets: macOS/arm64 tripped the post-process
h5dump NaN check (476AA3A4, "AMR -> 3D -> static block"), while
Linux/x86-64 read finite garbage and passed with a silently-wrong
overlay. The simulation's own restart reader (s_read_amr_restart) was
already correct and is the reference this fix mirrors.
Fix:
- post reader consumes the level field and uses the matching offsets.
- the 7-int header size is single-sourced as amr_restart_blk_hdr_ints
in m_constants and used by the writer and both readers, so the layout
cannot drift again.
- add a fail-closed header sanity check (level >= 1, integral
refinement of the coarse footprint) so any future drift aborts with a
clear message instead of emitting a corrupt overlay.
Verified on macOS/arm64 GNU+MPI: 476AA3A4 and all 54 AMR + restart-
roundtrip + MPI-consistency tests pass under --test-all (no-debug), 53/54
under reldebug. The single reldebug failure (D731AB7A, cont_damage
hypoelastic AMR) is a pre-existing snan-exposed NaN that reproduces
identically without this change and is unrelated to restart I/O.
Brings up/mega up to date with MFlowCode/MFC master (9 commits), clearing the PR's merge-conflict state so CI can run. Conflicts resolved (all preserve both branches' intent): - m_rhs.fpp / m_time_steppers.fpp (master MFlowCode#1650, grind timing): master moved per-stage timing from a cpu_time running-mean in s_compute_rhs to a system_clock minimum in the stepper, and dropped time_avg from the s_compute_rhs signature. Kept up/mega's active_box + rank-timing; adopted master's timing. Reconciled the AMR advance chain: the fine advance runs inside the stepper's new per-stage bracket, so the now-dead time_avg pass-through (s_amr_fine_stage_advance / subtree / children / subcycle -> s_compute_rhs) is removed rather than threaded to the old signature. - m_mpi_common.fpp / post/m_start_up.fpp (master #597f3c5c4, chemistry temperature seam-ghost exchange): kept up/mega's chem_T_comm rename and the AMR condition (chem_params%diffusion .or. amr); master's broadening of the post_process branch to all chemistry was already present in up/mega, so the resolution is a superset of both. Verified on macOS/arm64 GNU+MPI (--test-all): AMR 54/54, restart-roundtrip 2/2, MPI-consistency 3/3, chemistry 13/13, and the restart-header fix (476AA3A4) all pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An opt-in, default-off family of performance features and the measurement infrastructure they rest on. With all flags at their defaults the only touched production path is
s_mpi_decompose_computational_domain, refactored through the newm_boxmodule (byte-identical; covered by the existing suite).m_box(partition arithmetic),m_load_weight/load_weight_wrt(per-cell load-weight field + imbalance metric),m_sfc_partition/sfc_partition_wrt(Morton-SFC predicted-imbalance diagnostic),m_load_balance/load_balance(weighted static decomposition at init; AMR-fine-work-aware),m_rank_timing/rank_time_wrt(per-rank compute-time diagnostic).m_active_box/active_box: restricts reconstruction/Riemann/RK windows to a light-cone-grown box around non-ambient flow; strict-subset golden-tested.hybrid_wenoandhybrid_riemann(+hybrid_weno_eps,hybrid_smooth_flux): linear-optimal weights / central-or-Rusanov flux in smooth cells, full WENO/HLLC at flagged discontinuities (Jameson sensor, stencil-dilated, per-level under AMR).m_amr+m_amr_registers: two-level 2:1 refined block hierarchy; conservative restriction and conservative-linear prolongation with physics-specific closures; per-stage flux registers with Berger–Colella refluxing; Berger–Rigoutsos multi-block dynamic regrid; optional dt/2 subcycling; multi-rank (blocks may span rank seams; fine halo exchange; distributed registers); restart (both IO modes, regridded-layout persistence); AMR-aware post-processing (fine blocks visualizable as Silo overlay domains); GPU-resident fine level on both OpenACC and OpenMP offload.Full algorithm and user documentation:
docs/documentation/amr.md(support matrix enforced at runtime by the checker — unsupported combinations abort with named messages, never silently).AMR physics support matrix (abridged; authoritative table in amr.md)
Supported and golden-tested: single- and multi-fluid (5-eq,
mpp_lim) · 6-eq with per-block pressure relaxation · viscous (refluxed) · phase change (relax) · chemistry incl. species diffusion · Euler–Euler bubbles (polytropic/non-polytropic, mono/polydisperse, QBMM incl. non-polytropic with per-blockpb/mvside-state; dynamic regrid + subcycle) · acoustic sources (coarse-grid support with regrid exclusion) · immersed boundaries (multi-body, static or prescribed-motion, incl. dynamic regrid with body-containment expansion and per-substage guards) · 2D axisymmetric (per-block WENO-coefficient recompute) · stretched grids (exact parent-bisection ghost coordinates + per-swap coefficient recompute) · hybrid WENO/Riemann sensors (per-level) · Lagrangian bubbles (cloud excluded from blocks; two-way coupling on the coarse grid; regrid clips around the moving cloud) ·active_box(blocks contained in the growing window; agrees with plain AMR to ~1e-14) · IGR (restriction-only coupling: fine sigma solve seeded/Dirichlet-bounded by the coarse solve; documented truncation-order seam, exact free-stream) · 1D MHD/RMHD (div(B)=0 by construction in 1D; HLL and HLLD, incl. relativistic).Gated with named aborts (documented rationale): surface tension (seam force imbalance is structural — three fixes attempted and diagnosed in amr.md) · 2D/3D MHD (attempted and measured: the c/f seam is a continuous O(1) div(B) source GLM cleaning cannot remove — needs constrained-transport-class B prolongation/reflux) · hyperelasticity · 3D cylindrical (global azimuthal filter) · force-driven IB (
moving_ibm=2) · STL bodies · Riemann-extrapolation BCs (bc=-4) ·amr_subcycleunder IGR · stretched grids with Lagrangian/IB-regrid (uniform-spacing index conversions).Validation evidence
Known issues (all non-gating or in progress)
continue-on-error): an intermittent post-detected NaN on the two Lagrangian+AMR goldens. Exhaustively unreproducible off GitHub's runners — the exact failing stack (NVHPC 24.3 SDK,-tp=px -Kieee, HPC-X MPI, and the CI docker image itself under apptainer) passes elsewhere, as do native/zen2 builds; 24.5+ green. Documented at the golden definitions.Review guide
The commit history is arc-ordered (active-box → load-weight → SFC → weighted decomposition → rank timing → hybrid → m_box → AMR rungs → physics envelope → CI/GPU hardening); reviewing by arc is much easier than by file. The AMR arc builds stepwise: static hierarchy → restriction/prolongation → fine advance → refluxing → regrid → subcycling → multi-rank → GPU → each physics rung with its own validation. Commit messages carry the validation evidence for their change (measured defects, golden UUIDs, repro details for CI fixes).
All parameters ship default-off with
case_validatorentries, runtime checker gates, andcase.md/amr.mddocumentation.