Add major-chemistry data product (one feature per well) - #81
Merged
Conversation
New ogc_major_chemistry product: nm_major_chemistry. One GeoJSON feature per well, with each major-ion analyte's latest value/units/date plus well depth as properties. - backend/persisters/ogc_features.py: dump_major_chemistry_collection pivots per-(well,analyte) SummaryRecords into one feature per well, keyed (source, id); carries well_depth and geometry. - orchestration/assets/products.py: _MAJOR_CHEMISTRY (classic 8 major ions); products now unify over a list of parameters (single-parameter products run once, major-chemistry runs once per analyte and accumulates); source keys = union of the analytes' agencies; combine picks the dumper by output_type. - die_config.get_config: optional parameter override; treat ogc_major_chemistry as summary mode. - definitions.py: register ogc_major_chemistry as a supported output type (gets its own per-product job + schedule automatically). - products.yaml: nm_major_chemistry entry (all NM sources). - tests: pivot + geometry/required-field coverage for the new dumper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Your pull request is automatically being deployed to Dagster Cloud.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New
ogc_major_chemistryproduct —nm_major_chemistry. Produces one GeoJSON feature per well, with each major-ion analyte's latest value, units, and date plus well depth stored as feature properties.Major-ion suite (classic 8): calcium, magnesium, sodium, potassium, bicarbonate, carbonate, chloride, sulfate.
How it fits the per-source asset graph
Same
sources → combine → geoservershape as other products, with two extensions:parameter_name). Source keys for the product = the union of the analytes' agencies.dump_major_chemistry_collectiongroups per-(well, analyte)SummaryRecords by(source, id)into one feature per well:<analyte>,<analyte>_units,<analyte>_dateproperties, pluswell_depth/well_depth_unitsand point geometry.Changes
backend/persisters/ogc_features.py:dump_major_chemistry_collection.orchestration/assets/products.py:_MAJOR_CHEMISTRY,_product_params, union source-keys, multi-param source asset, output-type-based dumper selection.orchestration/resources/die_config.py: optionalparameteroverride; treatogc_major_chemistryas summary mode.orchestration/definitions.py: registerogc_major_chemistry(auto-gets its own job + schedule).orchestration/config/products.yaml:nm_major_chemistry(all NM sources, daily 11:00).tests/test_persisters/test_ogc_features.py: pivot + geometry/required-field coverage.Notes for reviewer
dg check defsloads all defs; the product expands to 5 source assets (bor, nmbgmr_amp, nmed_dwb, nmose_isc_seven_rivers, wqp) → combine → geoserver, plusnm_major_chemistry_job. 11 persister tests pass.source_pair → None), so missing analytes simply don't appear in a well's properties.main; independent of the open docs PR Improve in-code documentation for product asset graph #80 (no overlap beyond files both touch — this branch carries its own coherent versions).🤖 Generated with Claude Code