fix: validate Geo filter arguments before they reach the query string - #724
Conversation
vishal-bala
left a comment
There was a problem hiding this comment.
The injection is closed, and the reasoning in the description holds up. I reproduced the original hole against Redis 8.4.5 (search module 80410): a crafted latitude gives a UNION root under DIALECT 2 and leaks 3 documents where 1 is correct, and this branch refuses it at the constructor. A separate pass tried nineteen bypasses of the new guard, including numbers.Real subclasses overriding __str__/__repr__/__format__, a nondeterministic __float__ aimed at the range check, __index__-only integrals, numpy scalars, and str subclasses overriding lower(). Every one either rendered a plain numeric literal or raised. Validating in GeoSpec.__init__ is also the right layer: the spec holds all three arguments before any field is in scope, so the constructor is the only place that can own the check.
Two things should change before this merges, and the first is the argument the description is most confident about.
The radius is coerced but never bounded, and %i truncates it. That combination turns a documented value class into silently wrong results. Point 3 of the description is right that the guard should not rest on a format specifier, but the fix makes the constructor accept floats that %i cannot render. Measured against a document planted roughly 1.5 km north of the centre:
| Caller writes | Renders as | Server returns |
|---|---|---|
radius=1.9, unit="km" |
1 km |
1 hit, losing the 1.5 km document |
| the query that was asked for | 1.9 km |
2 hits |
radius=0.5, unit="km" |
0 km |
Invalid GeoFilter radius |
radius=-5 |
-5 km |
Invalid GeoFilter radius |
The first row is the one that matters: no exception, no warning, just a narrower result set. RediSearch accepts a fractional radius, so this is the template discarding precision rather than a server limit.
Second, rejecting Decimal is an unflagged behaviour break. Geo("l") == GeoRadius(Decimal("-122.4194"), Decimal("37.7749"), 1) renders @l:[-122.4194 37.7749 1 km] on main and raises TypeError here. Num already rejected Decimal, so this is Geo-only, and #721 treated the same rejection as a breaking change and said so under ## Release Notes. This one needs either the same note or a widened guard.
Everything else is optional. Longitude range, the unit list, case-insensitive unit handling, the inclusive endpoints and the DIALECT 2 premise all check out against the documentation and the server, and the LONGITUDE_RANGE/LATITUDE_RANGE endpoints mirror the query parser exactly (@loc:[180 0 3 km] parses, 180.00001 does not).
c584314 to
54271a0
Compare
|
Taking this over with Joo Bin's permission so it can ship today. Rebased onto The rebase needed Both must-fix items are closed. The radius no longer truncates. The radius is bounded: zero, negative and infinite are refused at the constructor.
The optional items are in too, plus four things the second review round turned up:
Two corrections to my own review, both found by mutation testing rather than reading. I was wrong to call And the All eleven guards are mutation-checked by reverting each alone. The integration geo filter uses a fractional radius, so a truncation regression fails against a real server — verified: reverting the template turns it red with |
`Geo.OPERATOR_MAP` rendered the radius with `%i`, which truncates toward zero. A 1.9 km radius silently queried 1 km, and 0.5 km rendered `0`, which the server answers `Invalid GeoFilter radius` -- so a sub-unit radius was inexpressible. Measured on 8.4.5 against a document 1.5 km out: `1 km` returns 1 hit, `1.9 km` returns 2. The radius now renders with `%s`, which is safe because `GeoRadius` coerces it to a builtin first. That coercion was already there and was already the real type guard; `%i` only looked like one by raising `TypeError` on a `str`. An integral float renders as an int, because `repr` switches to exponent form at 1e16 and `1e+16` is a syntax error at DIALECT 1, the server default. The radius is also bounded now. Zero, negative and infinite are refused at the constructor, where zero and negative previously built a query the server rejected and infinite escaped as an `OverflowError` from inside `Geo.__str__`. The bound is a chained comparison rather than `math.isfinite`, which raises `OverflowError` on an int too large to convert -- a radius has no upper bound to reject such a value first, unlike a coordinate. Four smaller corrections to the same change: - `_coerce_to_number_within` compares the range before `math.isfinite`, so an unconvertible int leaves as the documented `ValueError`, and `float()` is guarded for a `Real` that is finite but unrepresentable. - `GEO_UNITS` is an annotated tuple, so the allowlist is not mutable shared state and a subclass can still widen it. - A non-string unit leaves as the documented `ValueError`, found by lookup rather than by catching, so an `AttributeError` from inside a caller's own `lower()` still propagates. - The pole limit moves to the `Geo` class docstring, where it covers every geo query rather than contradicting the "-90 to 90" bullet it sat in. Every guard is mutation-checked by reverting it alone, both `OPERATOR_MAP` templates separately. The integration geo filter now uses a fractional radius, so a truncation regression fails against a real server. Co-authored-by: limjoobin <joobin.lim@redis.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 54271a0. Configure here.
54271a0 to
4581570
Compare
The integral-float normalisation says `repr` switches to exponent form and that DIALECT 1 rejects it, without saying that this is true of a positive exponent only. Measured on 8.4.5, `@loc:[-122.4194 37.7749 1e-05 km]` parses at both dialects while `1e+16` is a syntax error at DIALECT 1, so a small radius needs no treatment and reaching for one would be wasted work.
|
🚀 PR was released in |

Fixes #723
Summary
GeoSpecvalidated only itsunit.longitudeandlatitudewere stored as given despite theirfloatannotations, and every one of the four arguments is interpolated into a RediSearch query byGeo.__str__.A
strcoordinate carrying a]closed the geo clause and had its remainder parsed as syntax. Similar to #721, an injected|binds looser than the implicit space-AND, so it lifts to the root of the parse tree and any filter sharing the query stops constraining it:The
unithad the same shape by a different route, and this one is not in the issue as filed.unit.lower()not inself.GEO_UNITSis an equality test the caller controls, whileself._unit = unit.lower()stored the caller's object. Astrsubclass is neutralised bystr.lower()returning a builtin, but an object whoselower()returnsselfand whose__eq__matches"km"passed the check and then rendered its own__str__:Coordinates were also unbounded, so GeoRadius(9999, -9999, 1) and a NaN coordinate both rendered queries the server rejects, failing far from the line that caused them.
Changes
GeoSpec.__init__routes both coordinates through_coerce_to_number_within, which coerces to a builtin and requires a finite value inside an inclusive range —LONGITUDE_RANGE = (-180.0, 180.0)andLATITUDE_RANGE = (-90.0, 90.0), class attributes beside the existingGEO_UNITSso the numeric and unit domains share a namespace and a subclass can widen either.Constructor-time rather than render-time is deliberate. The error names the offending argument at the caller's line instead of surfacing from inside
__str__, andstr()is on the query path so it should not re-validate on every call. The bounds are inclusive because the antimeridian and the poles are real places.isfiniteis checked separately rather than left to the range comparison, so an infinite bound could not admit an infinite value. That much is unreachable through this class's own finite ranges and is covered by a subclass test. The ordering is load-bearing for a second and entirely reachable reason, though:isfiniteraisesOverflowErroron an int too large to convert to a float, so the range is compared first and rejects10**400as the documentedValueError._canonical_unitreturns the matched element ofGEO_UNITSrather than the value that matched it. The caller's__eq__still decides whether a unit matches; it no longer supplies what renders. There is deliberately noisinstance(unit, str)check, which would reject a legitimatestrsubclass — canonicalisation makes the identity oflower()'s return value irrelevant, since it is only ever compared, never rendered.GEO_UNITSstays the single source of truth for both the lookup and the error message.Unit validation still runs before the coordinates, so a caller passing both a bad unit and a bad coordinate sees the error they saw before.
%iwas doing two jobs badly. As a type guard it worked by accident, raisingTypeErroron astr, which left a format specifier load-bearing where nothing documented it. As a formatter it was simply wrong: it truncates toward zero, so a fractional radius silently queried a smaller circle than the caller asked for, and a sub-unit radius rendered0, which the server rejects outright.Measured on 8.4.5 against a document 1.5 km from the centre:
radius=1.9, unit="km"1 km1.9 kmradius=0.5, unit="km"0 kmInvalid GeoFilter radius0.5 kmSo the radius now renders through
%s, andGeoRadiuscoerces it — that coercion is the type guard, explicitly, with a test that fails without it. An integral float renders as an int, becausereprswitches to exponent form at 1e16 and1e+16is a syntax error at DIALECT 1, the Redis 8 server default.The radius is also bounded now: zero, negative and infinite are refused at the constructor. Measured, the server answers
Invalid GeoFilter radiusto zero and to a negative, and an infinite radius previously escaped as anOverflowErrorfrom insideGeo.__str__. The bound is a chained comparison rather thanmath.isfinite, which raisesOverflowErroron an int too large to convert — a radius has no upper bound to reject such a value first, unlike a coordinate.Num._coerce_numericfrom #721 becomes the module-level_coerce_to_number, becauseGeoSpecis not aFilterFieldand could not inherit it.Num._coerce_numericstays as a one-line delegator bindingcls.__name__, so error text is unchanged andTimestampstill reports its own name throughcls.Tests
30 geo cases in
tests/unit/test_filter.py, sharing a_geo_radius(**overrides)builder so each row names only the argument it changes._StrOverridingInt/_StrOverridingFloatare reused from #721 rather than duplicated per filter type. Every guard was mutation-checked by reverting it alone, including bothOPERATOR_MAPtemplates separately, since__ne__is otherwise asserted nowhere.The geo filter in
tests/integration/test_query.pynow uses a fractional radius, so a truncation regression fails against a real server rather than only against a string comparison. Verified: reverting the template alone turns that test red withInvalid GeoFilter radius.Release Notes
Geo filter arguments are now validated where they are supplied, rather than reaching the query string unchecked. A
strcoordinate could previously close the geo clause and have its remainder parsed as RediSearch syntax, so a filter built from untrusted input could be widened past the scope it was meant to enforce. Upgrade if any geo filter argument in your application originates from user input.One change moves results silently and deserves attention before you upgrade. A fractional radius used to be truncated toward zero, so
GeoRadius(lon, lat, 1.9, "km")queried a 1 km circle; it now queries 1.9 km and returns the rows it always should have. Nobody gets an error from this, only different rows.Three changes are backwards-incompatible, all of them turning a value that used to build a broken query into an error at the line that supplied it.
Decimalcoordinates now raiseTypeError, having rendered successfully before, so a coordinate arriving from aNUMERICdatabase column or fromjson.loads(payload, parse_float=Decimal)has to be passed throughfloat()first. A zero, negative or infinite radius now raisesValueError, where zero and negative previously built a query the server rejected withInvalid GeoFilter radiusand infinite raisedOverflowErrorfrom inside the formatter. And a coordinate outside its range now raisesValueErrorinstead of building a query the server refuses.