Restore lake cover lost to the NN fill, and fix the dateline seam in the CORE2 mask - #58
Merged
Merged
Conversation
Reconciling the OpenIFS mask with the FESOM mesh makes a cell land wherever the mesh has no wet node under it, and fill_flipped_from_nearest_neighbour then rebuilds its whole surface column from the nearest stable land cell. Lake cover is only another surface field to that fill, so a cell the input file called a lake inherits a dry neighbour's cl and stops being one. Measured on TCO95/CORE3, of 319 cells flipped from ocean to land, 70 carried cl >= 0.5 in the pristine input, including the whole Caspian (27/27) and the Great Lakes (17/17). After the flip none of them do. lakes.restore_flipped_lakes (default off) puts the pristine cl, dl and FLake prognostics back at every flipped cell. One rule, no threshold: cl is the lake share of a box and is capped by its water share -- cl <= 1 - lsm holds at all 40320 cells -- so the input file has already classified each cell. Restoring its own value gives the genuine lakes their mean 0.799 and the coastal cells, whose water ECMWF calls sea rather than lake, their 0.019. A cl >= 0.5 threshold writing 1.0 would overstate lake area at the former by 20 %, since 27 of the 70 sit between 0.5 and 0.7. dl is clipped to [2, 50] m, the band FLake actually integrates (flakeene_mod.F90:211-212,238 and surftstp_ctl_mod.F90:882), so the file says what the model will do rather than carrying bathymetry FLake will discard. The guard is on field presence, not on cycle number. CY43R3 was expected to lack the FLake fields; measured against ICMGGh6mvINIT_CORE2 it carries all of cl, dl, lmlt, lblt, lmld, lict, licd, ltlt and lshf, exactly as CY48R1 does, so a cycle gate would have disabled the feature where it in fact works. CycleSpec.lake_fields remains as the place a future cycle without FLake would be declared. This does not touch lsm, so all four OASIS masks are unaffected, and it does not remove the cells from the LPJ-GUESS gridlist, which is built from lsm alone. The soil-type repair is still required and is not substitutable by it.
read_fesom_grid_polygon locates each OpenIFS point in the FESOM triangulation in two passes: non-dateline triangles in standard coordinates, then dateline-crossing ones with longitudes shifted to 0-360. The shift does not straighten every triangle. Polar-cap triangles remain wide, because at the pole longitude carries no information and the three nodes can sit at any three longitudes: CORE2 has one at lat 89.8-90.0 with longitudes 99, 173 and 336, spanning 237 degrees after the shift. Laid out in the plane it crosses two thirds of the domain and overlaps almost every other triangle, and the trapezoid map behind the point-location index cannot be built from overlapping triangles. It raised "Triangulation is invalid" and the entire dateline pass was lost. That was caught by a bare except that printed a warning and continued, so the points the pass would have classified kept the "land" default and the mask was written anyway. Measured on TCO95/CORE2 the result was all 192 cells within 0.5 deg of lon 180 marked land, against 9 % in the ECMWF input and 3 % one degree away: a wall of false land from pole to pole, spread across every latitude band, 175 of the 396 ocean-to-land flips. Two changes. Triangles still spanning more than 180 degrees after the shift are dropped before the triangulation is built -- a triangle that cannot be drawn in the plane cannot locate points in it either, and the pole is covered by Pass 1. And a dateline pass that fails now raises instead of warning, because continuing writes a corrupt mask and the warning was not noticed. CORE2 goes from 192/192 to 17/192 land at the dateline, matching its input exactly, with the land total 11576 -> 11400. CORE3 has such a triangle too and happened to survive the trapezoid map; dropping it leaves its mask bit-identical, 11764 land, 0 cells changed. So this was never CORE2-specific, only a CORE2 symptom. No published file is corrected by this. The CORE2 ICMGGs in the pool still carry the seam and have to be regenerated or repaired separately; nothing has run on them. Also adds tools/repair_lake_cover.py, the in-place repair used to restore the lake fields to the published _v2 ICMGGs without recomputing any coastline.
LakeConfig was used as a string annotation without being imported, which flake8 reports as an undefined name and which fails the CI syntax gate. Import it alongside OCPConfig and drop the quotes. Also rewraps a comment whose second line began "# type:", which flake8 parses as a type comment and rejects as a syntax error. That one predates this branch and already fails on master, so the gate cannot pass without it.
Corrects only the cells in the dateline band whose land-sea value disagrees with a mask recomputed under the polar-cap guard. Recomputing today also disagrees away from the band -- 1 cell on TCO95/CORE2, 36 on TL255/CORE2 -- because the published files do not all date from the current mesh.nc. That is coastline drift rather than this defect, and folding it in would move a coastline silently under cover of a bug fix, so the band is the whole fix. A cell turned back to ocean cannot keep its land surface column, or it carries soil moisture, vegetation and a land skin temperature into a sea point. Each repaired cell is therefore rebuilt from the nearest stable ocean neighbour, the same great-circle rule the flip itself uses, with donors excluding the repaired set. Verifies before writing that every change lands inside the repair set, and aborts rather than producing a file otherwise. Applied to the four affected 48r1 pool files: ICMGGab45INIT_CORE2, _v2 192/192 -> 17/192 land, 11576 -> 11401 ICMGGabl7INIT_CORE2, _v2 248/258 -> 20/258 land, 25785 -> 25557 Lake cover and soil type came through untouched: cl >= 0.5 at 70 and 180 cells respectively, slt unchanged. ICMGGab45INIT_CORE2ice and its _v2 carry the same seam but are left alone: CORE2ice is a cavity mesh and no mesh.nc for it exists under input/fesom2, so no corrected mask can be built. The 43r3 ICMGGaackINIT_CORE2ice shares the defect and the same land count, so the bad mask was copied across cycles.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two silent surface-field defects, found while auditing what the land-sea flip does to a grid box.
1. The flip destroys lake cover
fill_flipped_from_nearest_neighbourrebuilds every flipped cell from the nearest land neighbour, andclis just another surface field to it — so a cell the input called a lake inherits a dry neighbour's cover.TCO95/CORE3: of 319 cells flipped to land, 70 had
cl >= 0.5in the input, including the whole Caspian (27/27) and Great Lakes (17/17). After the flip, none do. TCO319 loses ~1000 per file.lakes.restore_flipped_lakes(default off) restores the pristinecl,dland FLake prognostics at every flipped cell. No threshold is needed:clis the lake share of a box, capped by the water share (cl <= 1 - lsmat all 40 320 cells), so the input has already classified each cell — genuine lakes get their mean 0.799, coastal cells whose water is sea get their 0.019.dlis clipped to[2, 50]m, the band FLake actually integrates (flakeene_mod.F90:238).The guard is on field presence, not cycle number: CY43R3 was expected to lack the FLake fields but measurably carries all nine, so a cycle gate would have disabled the feature where it works.
This does not touch
lsm, so the OASIS masks are unaffected, and it does not remove cells from the LPJ-GUESS gridlist — the soil repair is still required and not substitutable (overlap: 67 of 314 cells).2. A wall of false land down the dateline
Pass 2 shifts dateline triangles to 0–360, but polar-cap triangles stay wide — at the pole longitude carries no information, and CORE2 has one spanning 237° after the shift. It overlaps most of the domain, so the trapezoid map cannot be built,
Triangulation is invalidwas raised, a bareexceptwarned and continued, and every point the pass would have classified kept thelanddefault:A pole-to-pole seam, accounting for 175 of the 396 ocean-to-land flips.
Fix: drop triangles still spanning >180° after the shift, and raise instead of warning when the pass fails.
CORE3 has such a triangle too and merely happened to survive, so this was never CORE2-specific.
Also
tools/repair_lake_cover.py— in-place repair of the published_v2ICMGGs that never recomputes a coastline; verifieslsmandsltstay bit-identical and aborts rather than writing if not.The last commit fixes flake8
F821(mine) andF723(pre-existing on master, so the syntax gate cannot pass without it).Testing
5 new tests driven by the real TCO95 ICMGG, including the
cl <= 1 - lsmconvention the restore depends on. Full suite 55 passed; CORE3 mask regression 0 cells changed.