Reorganize into a multi-domain toolkit (geo, storage, weather) - #1
Merged
Conversation
Restructure the package into storage/, geo/, weather/, logging and smoothing modules, each with heavy dependencies behind an optional extra (storage, geo, smoothing). Extract geodata utilities (swisstopo API client, raster handling, terrain analysis) and generic helpers (context-aware logging, 1D smoothers) so they can be shared across projects. Terrain slope computation is reimplemented on numpy.gradient to drop the torch dependency. Reconcile get_s3_client to accept optional credentials that fall back to the S3_ACCESS_KEY_ID / S3_SECRET_ACCESS_KEY environment variables, and keep the historical top-level exports resolvable lazily via PEP 562.
Cover raster IO, terrain slopes/masks, the swisstopo client (mocked HTTP), 1D smoothers and context-aware logging, and add get_s3_client credential and backwards-compatibility tests. Run ruff and pytest on Python 3.11 and 3.12 via GitHub Actions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns
simlab-toolsinto a proper shared tooling repo used across SIM lab projects, extracting reusable utilities frompIserables.What changed
storage/— S3 transfer helpers (moved from top-levels3.py).geo/—swisstopoAPI client,rasterIO,terrainslope/mask, sharedtypes.weather/— documented placeholder for observation + forecast loaders (no loaders yet).logging,smoothing— generic context-aware logging and 1D curve smoothers.get_s3_clientreconciled:key_id/key_secretnow optional, falling back toS3_ACCESS_KEY_ID/S3_SECRET_ACCESS_KEY. Fixes the drift where callers used the one-arg form. Top-levelfrom simlab_tools import get_s3_clientstill works (resolved lazily).terrainreimplements slope gradients onnumpy.gradient, dropping thetorchdependency.storage/geo/smoothing/weather/all/dev;moto/pytestmoved out of runtime deps; ruff config added;uv.lockregenerated.get_s3_client; new GitHub Actions workflow runs ruff + pytest on Python 3.11 and 3.12.Verification
ruff checkclean and 59 tests pass in a fresh 3.12 venv (uv pip install -e ".[dev]").Note
The companion
pIserablesPR (which consumes this via the git dependency) should be merged after this one.