Skip to content

Share Redis client across rate-limit policy instances #2143

@renuka-fernando

Description

@renuka-fernando

Description

The rate-limit policy (advanced-ratelimit, used directly and via basic-ratelimit) creates a brand-new redis.Client — a full connection pool — per policy instance in GetPolicy, and again on every config reload. Each is pinged and never closed.

At scale this does not scale: with many routes/APIs each attaching a rate-limit policy, the gateway opens one connection pool per route per replica, and every config reload orphans the old pools (a slow leak). This is a pre-existing issue in the redis backend, independent of any newer backend.

Improvement

Introduce a process-wide Redis client registry: get-or-create a single shared *redis.Client per distinct connection configuration (address, db, auth, timeouts, pool size), ping it once on creation, and reuse it across all policy instances and reloads. Add a redis.poolSize setting so operators size the one shared pool deliberately instead of inheriting N_instances × default.

This collapses Redis connections from O(routes × replicas) to one pool per Redis endpoint per replica, and removes the per-reload leak. It benefits both the redis and redis-local-async backends.

Scope

  • advanced-ratelimit policy GetPolicy client creation (and basic-ratelimit via delegation).
  • Applies whenever backend = redis or backend = redis-local-async.

Tasks

  • Process-wide client registry keyed by connection config (addr, user, password, db, timeouts, pool size)
  • GetPolicy gets-or-creates from the registry instead of redis.NewClient; ping once on create, skip on reuse
  • Add redis.poolSize system parameter to the policy definitions
  • Tests: same config -> same client; differing config -> distinct; no re-ping on reuse

Performance results to be included in the PR.

Related Issue

Part of #2141

Metadata

Metadata

Labels

Area/GatewayAny issues related to API deployment in gateway, routing etc.Aspect/PerformanceLatency, optimizations, load issuesType/Improvement

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions