Description
The rate-limit policy (advanced-ratelimit, used directly and via basic-ratelimit) creates a brand-new redis.Client — a full connection pool — per policy instance in GetPolicy, and again on every config reload. Each is pinged and never closed.
At scale this does not scale: with many routes/APIs each attaching a rate-limit policy, the gateway opens one connection pool per route per replica, and every config reload orphans the old pools (a slow leak). This is a pre-existing issue in the redis backend, independent of any newer backend.
Improvement
Introduce a process-wide Redis client registry: get-or-create a single shared *redis.Client per distinct connection configuration (address, db, auth, timeouts, pool size), ping it once on creation, and reuse it across all policy instances and reloads. Add a redis.poolSize setting so operators size the one shared pool deliberately instead of inheriting N_instances × default.
This collapses Redis connections from O(routes × replicas) to one pool per Redis endpoint per replica, and removes the per-reload leak. It benefits both the redis and redis-local-async backends.
Scope
advanced-ratelimit policy GetPolicy client creation (and basic-ratelimit via delegation).
- Applies whenever
backend = redis or backend = redis-local-async.
Tasks
Performance results to be included in the PR.
Related Issue
Part of #2141
Description
The rate-limit policy (
advanced-ratelimit, used directly and viabasic-ratelimit) creates a brand-newredis.Client— a full connection pool — per policy instance inGetPolicy, and again on every config reload. Each is pinged and never closed.At scale this does not scale: with many routes/APIs each attaching a rate-limit policy, the gateway opens one connection pool per route per replica, and every config reload orphans the old pools (a slow leak). This is a pre-existing issue in the
redisbackend, independent of any newer backend.Improvement
Introduce a process-wide Redis client registry: get-or-create a single shared
*redis.Clientper distinct connection configuration (address, db, auth, timeouts, pool size), ping it once on creation, and reuse it across all policy instances and reloads. Add aredis.poolSizesetting so operators size the one shared pool deliberately instead of inheritingN_instances × default.This collapses Redis connections from O(routes × replicas) to one pool per Redis endpoint per replica, and removes the per-reload leak. It benefits both the
redisandredis-local-asyncbackends.Scope
advanced-ratelimitpolicyGetPolicyclient creation (andbasic-ratelimitvia delegation).backend = redisorbackend = redis-local-async.Tasks
GetPolicygets-or-creates from the registry instead ofredis.NewClient; ping once on create, skip on reuseredis.poolSizesystem parameter to the policy definitionsRelated Issue
Part of #2141