Skip to content

[get_paths] version="latest" resolves positionally to the first YAML entry, not the highest semver #73

Description

@bdestombe

Severity: minor (silent wrong-version selection; depends on manual YAML ordering)

Where: src/nhflodata/get_paths.py:85-86 (and, for contrast, the specific-version branch at :89-94).

Problem. get_abs_data_path(..., version="latest") resolves "latest" to rep[name][0] — the first entry of the dataset's list in repository.yaml — not the highest semantic version:

if version == "latest":
    version_index = 0

So "latest" is defined purely by manual ordering in the YAML, even though the docstring (L38-40) promises semantic version numbers. If a dataset's entries are ever appended in chronological (oldest-first) order, or otherwise not kept sorted newest-first, version="latest" silently returns an older dataset with no warning. The specific-version branch (:89-94) is order-tolerant — it matches version_nhflo by value — so only the "latest" path is affected, but "latest" is the common case.

Impact. The NHFLO/models 09pwnmodel2 script resolves all 10 of its datasets with version="latest" (modelscripts/09pwnmodel2/01_pwnmodel2.py:82-95bodemlagen_pwn_2024, oppervlaktewater_pwn_shapes_panden, nhi_chloride_concentration, wells_pwn, wells_tata, bodemlagen_pwn_regis_koppeltabel, lakes_pwn, drains_pwn, hfb_pwn, doorsnedes_nh), so the model silently depends on repository.yaml ordering for every dataset. Registry metadata is already known to be inconsistent for at least bodemlagen_pwn_regis_koppeltabel (self-referential previous_version, see #71), which makes trusting positional order fragile.

Suggested fix. Resolve "latest" by semantic version rather than position: sort a dataset's entries by version_nhflo (validate each with the existing is_valid_semver) and select the max, e.g.

if version == "latest":
    versions = [item["version_nhflo"] for item in rep[name]]
    version_index = max(range(len(versions)), key=lambda i: _semver_key(versions[i]))

Alternatively, validate at load time that each dataset's list is sorted newest-first and raise loudly otherwise. Either way, once "latest" is well-defined, consider having consumers (e.g. the 09pwnmodel2 script) pin explicit versions for reproducibility.


Filed from the 2026-07 systematic review of the NHFLO 09pwnmodel2 model stack (tracked in NHFLO/models#126). This is review finding M18 — the one finding from that review that had not yet been filed in any repo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions