simlab-tools

Shared research tooling for SIM lab projects: loading weather data, geodata, and transferring files to and from object storage. The package is organised into topical sub-packages, each keeping its heavy dependencies behind an optional extra so consumers install only what they use.

Installation

Install straight from GitHub, selecting the extras you need:

# Object storage only
uv add "simlab-tools[storage] @ git+https://github.com/simlab-vs/simlab-tools.git"

# Geodata utilities
uv add "simlab-tools[geo] @ git+https://github.com/simlab-vs/simlab-tools.git"

# Everything
uv add "simlab-tools[all] @ git+https://github.com/simlab-vs/simlab-tools.git"

Available extras: storage, geo, smoothing, weather, all, and dev (the full toolkit plus test/lint tooling).

Subpackages

simlab_tools.storage — object storage (extra: storage)

Read and write data to S3-compatible buckets (e.g. SwitchCloud): transfer files with progress bars and multipart transfers, or read/write Hive-partitioned Parquet datasets with polars for scalability. Credentials come from your ~/.aws/credentials profiles.

from simlab_tools.storage import get_s3_client, download_files_from_bucket

client = get_s3_client("https://zhw-a.s3.cloud.switch.ch", profile="switch")
download_files_from_bucket(client, "ofen", "dejection_cones_dem", "data/", file_extensions=[".tif"])

import pyarrow.dataset as ds
from simlab_tools.storage import get_s3_filesystem, write_dataset, read_dataset

fs = get_s3_filesystem("https://zhw-a.s3.cloud.switch.ch", profile="switch")
write_dataset(df, fs, "research-data", "measurements", partition_cols=["station", "year"])
recent = read_dataset(fs, "research-data", "measurements", filters=ds.field("year") >= 2024)

Credentials resolve from a named profile in ~/.aws/credentials, then explicit key_id / key_secret arguments, then the S3_ACCESS_KEY_ID / S3_SECRET_ACCESS_KEY environment variables. See storage/README.md for the full guide, including a local-CSV-to-partitioned-Parquet walkthrough.

For backwards compatibility, the storage functions are also importable from the top-level package: from simlab_tools import get_s3_client.

simlab_tools.geo — geodata (extra: geo)

• raster: load_and_merge_rasters, export_multiband_geotiff, convert_raster_to_xarray — load/merge raster tiles, write GeoTIFFs, wrap arrays as rioxarray DataArrays.
• terrain: compute_slope_components, create_geometry_mask — DEM slope gradients and polygon rasterisation.
• swisstopo: Point, BoundingBox, query_layer, query_layer_from_tiles, merge_river_segments — a client for the swisstopo / geo.admin.ch REST API.

from simlab_tools.geo import load_and_merge_rasters, compute_slope_components

mosaic, x, y, transform, crs = load_and_merge_rasters(["tile_a.tif", "tile_b.tif"])
slopes = compute_slope_components(mosaic, pixel_size=0.5)

simlab_tools.smoothing — 1D smoothers (extra: smoothing)

Interchangeable 1D curve smoothers behind a common smooth(x, y) protocol: MovingAverageSmoother, SplineSmoother, SavGolSmoother, APLRSmoother, and a get_smoother(method) factory.

simlab_tools.logging — context-aware logging

configure_logging, set_site_context and SiteFormatter attach a per-context key (e.g. a site or job id) to every log record, keeping logs attributable across concurrent workers.

simlab_tools.weather — weather data (planned)

Placeholder for provider-agnostic loaders of weather observations and forecasts. The public surface is documented in simlab_tools/weather/__init__.py; implementations are tracked as future work.

Development

uv pip install -e ".[dev]"
ruff check .
pytest

Continuous integration (ruff + pytest on Python 3.11 and 3.12) runs on every push and pull request via GitHub Actions.