Linearize curve geometries before exporting to WKT - #110
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #62.
Parcel/building sources loaded from Esri file geodatabases (GDB) can contain curve geometry types such as
MULTISURFACE,CURVEPOLYGON, andCOMPOUNDCURVEwhen the source data has curved boundary segments (arcs, circular street features, etc). This is common in some county-sourced building/parcel GDBs.ogr_source_to_csv()inopenaddr/conform.pyreads each feature's geometry with OGR and exports it directly to WKT viageom.ExportToWkt(). That WKT is later parsed with shapely'swkt_loads(). Shapely/GEOS does not understand GDAL's curve-geometry WKT types, so any GDB source containing curved geometry causes a downstream parse failure (reported asUnknown type: MULTISURFACEin the issue).Fix
In
ogr_source_to_csv, after reprojecting each feature's geometry, checkgeom.HasCurveGeometry()and if true, callgeom.GetLinearGeometry()(available since GDAL 2.0) before further processing. This converts curve types to their linear equivalents (e.g.MULTISURFACE->MULTIPOLYGON,CURVEPOLYGON->POLYGON) that shapely can parse, while leaving already-linear geometries (the common case for shapefile/GeoJSON/most GDB data) untouched.This function is shared by shapefile, GDB, GML/XML, and GeoKPG sources (
format_string in ("shapefile", "xml", "gdb", "gpkg")), so the fix is applied at the one call site that's actually reachable from GDB sources rather than scattered across everyExportToWkt()call in the file. The otherExportToWkt()call sites (geojson_source_to_csv,row_extract_and_reproject) operate on GeoJSON/CSV-sourced geometry, which cannot contain OGR curve types, so they're left alone.Verification
I don't have a real failing GDB file, so I reproduced this synthetically:
MULTISURFACE (CURVEPOLYGON (CIRCULARSTRING (...)))geometry directly withogr.CreateGeometryFromWkt(...)and confirmedwkt_loads(geom.ExportToWkt())raisesNotImplementedError: Nonlinear geometry types are not currently supportedon unpatched code — same failure class as the issue (shapely/GEOS choking on curve WKT)..gpkg, one of the formats sharing this code path with.gdb) containing a single feature with that same curve geometry, and ran it through the actualogr_source_to_csv()function end-to-end.MULTISURFACE (CURVEPOLYGON (...))) straight into the output CSV, which would later fail to parse downstream viawkt_loads.MULTIPOLYGON, parses cleanly withwkt_loads, and its bounds/area are sane (not degenerate) relative to the original curve geometry.test_ogr_source_to_csv_multisurfaceinopenaddr/tests/conform.py, that builds this synthetic GPKG fixture, runs it throughogr_source_to_csv, and asserts the output WKT contains noCURVE/SURFACEtype names, parses as aMultiPolygonvia shapely, and has sane bounds/area.openaddr.tests.conformsuite in the repo'sbatch-machine-test:latestdocker image: all 66 tests pass, including the new one.Caveat for reviewers: this is verified against a synthetic curve geometry (constructed directly with OGR, and via a synthetic GeoPackage fixture standing in for a GDB), not a real downloaded parcel/building GDB from one of the affected sources. The reasoning for why GeoPackage is a faithful stand-in is that both formats share the exact same
ogr_source_to_csv()code path and both can carry OGR curve geometry types end-to-end through the OGR API. If possible, it'd be worth testing against one of the real GDB sources referenced in the issue (e.g. Trempealeau County, WI) before/after this change.