Skip to content

Fix geofence queries: bind the fence, and test containment in Go - #400

Open
TurtIeSocks wants to merge 2 commits into
mainfrom
c/fence-sql-parameterize
Open

Fix geofence queries: bind the fence, and test containment in Go#400
TurtIeSocks wants to merge 2 commits into
mainfrom
c/fence-sql-parameterize

Conversation

@TurtIeSocks

@TurtIeSocks TurtIeSocks commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Problem

The three geofence queries built their statement by concatenating the fence's JSON into it:

"AND ST_CONTAINS(ST_GeomFromGeoJSON('"+string(bytes)+"', 2, 0), POINT(lon, lat))"

The fence arrives as a request body, and geojson.Feature round-trips its entire properties map through MarshalJSON. A property value is free-form text, so anything a caller puts there lands verbatim inside that quoted literal, including a quote of its own. /api/quest-status, /api/pokestop-positions and /api/clear-quests are all affected, along with the gin NormaliseFenceRequest path. All three require authentication, so this is not reachable anonymously.

Approach

FenceContainsPredicate is now a constant carrying a bind placeholder, and FenceQueryArgs assembles the bounding-box corners and the fence JSON in the order every one of these queries binds them. The statement text no longer varies with the fence, so its contents have no literal to escape.

Which rows match does not change. The same predicates run in the same order against the same values; only the transport differs, with the fence passed as data rather than as text. Each query keeps its own comparison operators, including the >/< in quest-status against the >=/<= in the other two.

The timeout, measured

I had assumed repeated GeoJSON parsing was the cost. That was wrong, and the measurement says so plainly. MariaDB 11.8, 1,000,000 pokestops with ix_coords(lat, lon), a 2000-vertex fence (83 KB of GeoJSON) whose bounding box holds 628,560 candidates:

Shape Time
Current query, fence inlined 55.0 s
SET @fence = ST_GeomFromGeoJSON(...), then ST_CONTAINS(@fence, ...) 57.6 s
The same through a CTE 57.8 s
WITH ... AS MATERIALIZED syntax error on MariaDB
Bounding-box scan alone, all 628,560 rows read 0.9 s
The same 628,560 points against a compiled fence in Go 3.3 s
Bounding box in SQL, containment in Go ≈ 4.2 s

Hoisting the parse out of the row loop changes nothing, and that is the result that matters: it rules out repeated parsing and leaves per-row ST_CONTAINS against a many-vertex polygon as the cost. Neither variant suggested in #398 moves the number, and one of them does not parse on MariaDB at all, since AS MATERIALIZED is PostgreSQL syntax.

So containment moves to Go. Candidates come from the bounding box, which the existing index already serves, and each one is tested against geo.CompileFence as the rows stream past, retaining only matches. That is the same matcher MatchGeofences already uses for stats and webhook area attribution, and it is differentially tested against orb.

Two consequences worth stating. Fences that are not polygons keep the SQL predicate, so exotic geometries are untouched. And CompileFence counts the polygon boundary as inside while ST_CONTAINS follows OGC and treats it as outside, so a fort exactly on a fence edge is now attributed the same way here as it already is everywhere else.

These are synthetic numbers on uniform random data in a container, not a production table, so treat the ratio as the finding and not the absolute times.

One note for anyone revisiting the SET @fence form regardless: user variables are connection-scoped and the handle is pooled (SetMaxOpenConns in main.go). These call sites use Select, Get and SelectContext, which each take an arbitrary connection, so the SET and the SELECT can land on different ones. @fence would then be NULL, ST_Contains(NULL, ...) would be NULL, and every row would filter out: zero results, no error.

Testing

Six tests in db/geofence_query_test.go. Three for the injection fix: the predicate keeps its placeholder and carries no literal to break out of, a hostile properties value survives only as a bound argument, and the bounding-box corner order the callers depend on. Three for the containment move: non-polygon fences fall back to SQL while polygons and multipolygons do not, the matcher's lat/lon argument order is right (the fences used have differing lat and lon spans, so a swap is visible), and boundary points count as inside.

go build -tags go_json ./... and golangci-lint run are clean, and the db, decoder and root suites all pass.

🤖 Generated with Claude Code

The three geofence queries built their statement by concatenating the
fence's JSON into it:

    "AND ST_CONTAINS(ST_GeomFromGeoJSON('"+string(bytes)+"', 2, 0), ...)"

The fence comes from a request body, and geojson.Feature round-trips its
whole properties map through MarshalJSON, so a property value containing
a single quote closes the string literal it lands in. That is SQL
injection on /api/quest-status, /api/pokestop-positions and
/api/clear-quests. All three require authentication, so it is not
reachable anonymously.

Move the predicate into the FenceContainsPredicate constant, which binds
the fence with a placeholder, and add FenceQueryArgs to assemble the
bounding-box corners and the fence JSON in the order every one of these
queries binds them. The statement text no longer varies with the fence,
so there is no literal for its contents to escape.

No change to which rows match: the same predicates run in the same
order, with the fence passed as data instead of as text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jfberry

jfberry commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

I don't think this actually moves the fence build out of the command (as per my message on the other issue).

Asking MariaDB to evaluate ST_CONTAINS for every candidate row is what
made large geofences time out. Measured on MariaDB 11.8 with 1,000,000
pokestops and a 2000-vertex fence whose bounding box held 628,560
candidates:

    current query, fence inlined       55.0 s
    SET @Fence, then ST_CONTAINS(@Fence)  57.6 s
    the same via a CTE                 57.8 s
    bounding-box scan, all rows read    0.9 s
    the same 628,560 points in Go       3.3 s

Hoisting the GeoJSON parse out of the row loop changes nothing, which is
what identifies per-row containment rather than repeated parsing as the
cost. So the fix is to stop asking SQL for containment: select candidates
by bounding box, which the ix_coords index already serves, and test them
against a compiled fence while the rows stream past.

geo.CompileFence is the same matcher MatchGeofences already uses for
stats and webhook area attribution, and is differentially tested against
orb. Only matching rows are retained, so nothing materialises the full
candidate set.

Fences that are not polygons keep the SQL predicate, so exotic geometries
behave exactly as before.

One deliberate behaviour change: CompileFence counts the polygon boundary
as inside, where ST_CONTAINS follows OGC and treats boundary points as
outside. A fort exactly on a fence edge is now attributed the same way
here as it already is in stats and webhooks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@TurtIeSocks TurtIeSocks changed the title Bind the geofence as a query parameter Fix geofence queries: bind the fence, and test containment in Go Aug 20, 2026
@TurtIeSocks

Copy link
Copy Markdown
Contributor Author

Split out of #398 so it can land on its own. Two separate things in here.

First, the fence was being concatenated into the SQL text rather than bound. A geofence body round-trips its whole properties map through MarshalJSON, so a property value containing a quote ends up inside the string literal. All three endpoints require auth so it isn't reachable anonymously, but it is on main today.

Second, the timeout, with the numbers in the description. Short version is that hoisting the GeoJSON parse out of the row loop changes nothing, so the cost is per row ST_CONTAINS against a many vertex polygon. Bounding box in SQL plus containment in Go takes it from 55 s to roughly 4 s, and it helps every instance rather than only the fort_in_memory ones. Non polygon fences keep the SQL predicate so nothing exotic changes.

@jfberry does this shape look right to you? Happy to split the two commits into separate PRs if you'd rather take the injection fix on its own first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants