Add :era5_land dataset support and unwrap zip-wrapped CDS responses - #9
Merged
Conversation
resolve_dataset() maps dataset=:era5/:era5_land to the right CDS product id and product_type (ERA5-Land's reanalysis-era5-land takes no product_type key), wired into hourly()/monthly()/yearly(). Some CDS datasets (reanalysis-era5-land observed in practice) wrap their netcdf output in a zip archive even when format=netcdf is requested. download_cds_file now detects the zip magic bytes and unwraps in place, so callers always get the requested format regardless of what the gateway actually sends. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes the Documenter missing_docs check, which flagged the new docstring as not included in any @docs block.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
resolve_dataset(dataset, pressure_levels)maps a newdatasetkeyword(
:era5, the default, or:era5_land) to the right CDS product id andproduct_type, wired intohourly(),monthly(), andyearly().reanalysis-era5-landtakes noproduct_typekey at all, unlike thesingle-levels/pressure-levels products.
download_cds_filenow detects and unwraps zip-wrapped CDS responses.reanalysis-era5-landwas observed in practice to sometimes return a ziparchive even when
format=netcdfis explicitly requested; previously theraw zip bytes were written straight to the output path, so any netCDF
reader downstream would fail on it. The fix checks the response for the
zip magic bytes and, if present, extracts the single member in place.
Motivation
Found while adding ERA5-Land support to NumericalEarth.jl
(NumericalEarth/NumericalEarth.jl#490): a live download of a full year of
hourly
reanalysis-era5-landdata succeeded at every CDS-side step butfailed downstream because one monthly chunk came back as a zip. This fixes
it at the source so any caller of
retrieve/download_cds_file— not justthat one downstream extension — gets the requested format regardless of
what the gateway actually sends.
Testing
New offline unit tests in
test/runtests.jl:resolve_datasetfor bothdatasets and the pressure-levels/land-with-pressure-levels error case, and
unwrap_zip_response!/is_zip_fileagainst a synthetic zip fixture and aplain (non-zip) file that should be left untouched.
🤖 Generated with Claude Code