Add a pytest suite covering the nhflotools code 09pwnmodel2 uses - #70
Merged
Conversation
The boundary conditions and the layer model of the active PWN model were untested: of the thirteen entry points 01_pwnmodel2.py imports, only interface_elevation and the Tata wells had any coverage. This adds 126 tests over that whole closure, running in about four seconds. Tests build their own DISV vertex grid (tests/util.py) instead of calling gridgen, so the unit tests need no executables, no network and no large datasets, and every cell area is an exact float that makes first-principles expected values possible. Live services -- HHNK, REGIS, RWS, background tiles -- are monkeypatched by name; the rest of nlmod runs for real, so the suite doubles as a compatibility canary for nlmod@dev. One mf6-marked test runs MODFLOW on 3x3x2 cells and checks budget closure plus an analytic head field, fetching the executable through nlmod.util.get_exe_path on first use. Every file was mutation-checked while being written: a plausible bug was introduced into the source, the test confirmed to fail, and the source restored. That covers the panden conductance formula reverting to resistance aggregation (a silent ~1e4x error), the polder fallback conductance being zeroed, the northsea guard being dropped, numeric_only being removed from the well median, extrapolate_ds's in-place contract breaking, the nearest-donor metric swapping axes, the budget threshold inverting, and the head field mirroring on the x axis. pyarrow becomes a runtime dependency: well.py reads a feather file via pandas, which needs it in production as well as in the tests. Also adds a test workflow keyed on the current nlmod@dev and nhflodata@main commits so the moving branches cannot serve a stale environment, a README overview separating the modules 09pwnmodel2 uses from the untested rest, and TEST_PLAN.md recording the design, the mutation evidence and the deviations. Ten defects found while writing these tests are filed as #60-#69 rather than pinned by a test, so none of them is entrenched as expected behaviour. The REGIS-completeness guard (#65) is covered by an xfail(strict=True) test that turns into a visible failure once the guard is fixed. Removes tests/test_hhnk.py and tests/test_nhflo_utils.py, which held no tests while suggesting hhnk.py and nhflo_utils.py were covered.
The variable was unset by the autouse hygiene fixture, which is function scoped and therefore runs after collection and after any session- or module-scoped fixture. Nothing resolves data paths that early today, so the suite did pass with the variable set, but the guard sat one scope too low to be relied on: a parametrisation or a session fixture that resolved a dataset path would have read the developer's mount instead of the packaged mockup. Moving it to pytest_configure closes that: it fires before collection, so every later mechanism sees the variable unset. pytest_unconfigure puts it back for in-process runners that outlive the session. get_abs_data_path resolves against the variable whenever it is set and only warns when the result is missing, so without this the module would pass or fail on whichever datasets happen to be mounted locally rather than on what NHFLO/data ships. Adds test_data_location_env_is_suppressed_for_the_session so removing the suppression fails with the cause named, instead of 41 opaque path failures. Verified by running the suite with the variable unset, set to an existing empty directory, and set to the real mockup root, and by removing the suppression with the variable set: 42 fail, led by the new test.
pytest-env was already a test dependency and already carried MPLBACKEND, so
the data location belongs there too:
env = ["MPLBACKEND=Agg", "NHFLODATA_LOCATION="]
pytest-env overrides an inherited value with the empty string, and get_paths
reads that as "use mockup" (get_paths.py:69 defaults the variable to "" and
:96 branches on it being falsy), so empty resolves exactly as unset does.
Being an ini setting it also applies before collection, which is what the
previous pytest_configure hook was there to guarantee -- one declarative line
replaces thirty of conftest.
The guard test now asserts the resolved path lies under the installed
nhflodata/data/mockup rather than asserting the variable is None. That checks
the outcome instead of the mechanism, so it keeps holding whichever way the
suppression is implemented, and it also fails if nhflodata ever stops treating
an empty value as "use mockup" -- which the previous test could not detect.
Verified with the variable unset, set to an existing empty directory, and set
to a nonexistent path: green in all three. With the pytest-env entry removed
and the variable set, 42 tests fail, led by the guard test.
Combine the suite's fixtures (vertex_ds/gwf_disv/hygiene) with main's _build_disv/disv_grid factory in conftest, and keep both the drn_from_waterboard_data tests and main's exclude-mask tests in test_polder.
nlmod@dev gave aggregate_by_weighted_mean_to_ds configurable source layer names, defaulting the floor to 'botm'; the NHI chloride file names it 'bottom'. Caught by the eight test_nhi_chloride tests on CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The boundary conditions and the layer model of the active PWN model were untested. Of the thirteen entry points
01_pwnmodel2.pyimports, onlyinterface_elevationand the Tata wells had any coverage; the RIV, DRN, GHB, CHD and WEL boundaries, the chloride initial condition, the layer model and the output loaders had none.This adds 126 tests over that whole closure, running in about four seconds.
Approach
Tests build their own DISV vertex grid (
tests/util.py) rather than calling gridgen, so the unit tests need no executables, no network and no large datasets — and every cell area is an exact float, which is what makes first-principles expected values possible instead of pasted implementation output.Live services (HHNK, REGIS, RWS, background tiles) are monkeypatched by name with the reason stated; everything else runs the real nlmod code, so the suite doubles as a compatibility canary for
nlmod@dev. Onemf6-marked test runs MODFLOW on a 3x3x2 model and checks budget closure plus an analytic head fieldh = (2x + y)/50, with deliberately unequal x and y coefficients so a mirrored or swapped axis in the.grb/.hdspath fails where a symmetric field would survive. It fetches the executable vianlmod.util.get_exe_path(download_if_not_found=True), so it works on a fresh runner without a preinstalled binary.test_nhflodata_contract.pytest_pwnlayers3_plot.pytest_postprocessing.pytest_nhi_chloride.pytest_pwnlayers3_layers.pytest_major_surface_waters.pytest_well.pytest_panden.pytest_polder.pytest_pwnlayers_get_top.pytest_mf6_smoke.pyEvidence the tests catch bugs
Every file was mutation-checked while being written: a plausible bug was introduced into the source, the test confirmed to fail, and the source restored. Among those caught:
numeric_onlydropped from the well median, and the split oversec_nputremovedextrapolate_ds's in-place contract broken, leaving NaNs under the seaget_top_from_ahnnearest-donor metric axis-swapped, and the falsy-constant sea guard introduced_compute_thickness_ratiosDefects found, filed rather than pinned
Ten issues were found while writing these tests and filed as #60–#69, so none is entrenched as expected behaviour by a test asserting it. Notable ones:
check_budget_discrepancyreads only the incremental budget frame, never the cumulative one — the transport model is transient, so a run whose per-step discrepancies each sit under the threshold can still accumulate past it and be reported as fine.Lengths must match to comparein exactly the missing-layer case it exists for, instead of the message naming the fix. Covered by anxfail(strict=True)test that becomes a visible failure once the guard is fixed.chd_ghb_from_major_surface_watersalways returnsts_sea=None(the series does attach tochd, so models are unaffected — the return value and docstring are wrong).concentration_meanis biased toward zero wherever concentration is NaN.Other changes
pyarrowbecomes a runtime dependency —well.pyreads a feather file through pandas, which needs it in production, not just in tests. It was missing.nlmod@devandnhflodata@maincommits, so those moving branches cannot serve a stale environment (the failure mode that produced theget_isosurface(left=...)breakage).TEST_PLAN.mdrecording the design, the mutation evidence, the deviations from plan, and the progress table.tests/test_hhnk.pyandtests/test_nhflo_utils.py, which contained no tests while suggestinghhnk.pyandnhflo_utils.pywere covered.Notes for review
TEST_PLAN.md.mainand independent of Stop storing idomain in pwnlayers/pwnlayers3 layer models #59 — the suite was verified green against bothmain's source and Stop storing idomain in pwnlayers/pwnlayers3 layer models #59's.