Skip to content

feat: add Redis connector - #514

Open
pushtisonawala wants to merge 2 commits into
DrDroidLab:mainfrom
pushtisonawala:redis-connector
Open

pushtisonawala wants to merge 2 commits into
DrDroidLab:mainfrom
pushtisonawala:redis-connector

Conversation

@pushtisonawala

@pushtisonawala pushtisonawala commented Sep 10, 2026

Copy link
Copy Markdown

Fixes #480

Adds a Redis source to Playbooks. Playbooks had no way to connect to a Redis
instance before this; the only Redis-related asset was a CloudWatch-based text
template.

Demo (90s)

https://www.loom.com/share/902b71f03a6e4b85aa001939e5390a08

What it does

Three tasks, following the existing Postgres / Bash connector pattern:

Task Redis call Result
Fetch Redis INFO stats INFO [section] table of metric → value
Fetch Redis slow query log SLOWLOG GET n table (id, start_time, duration_us, command, client)
Run a read-only Redis command arbitrary command table

Adds a Redis source to Playbooks with three tasks:
- Fetch Redis INFO stats (optional section)
- Fetch Redis slow query log (SLOWLOG GET)
- Run a read-only Redis command (allowlist-guarded; write/admin commands blocked)

Covers issue DrDroidLab#480. Follows the existing Postgres/Bash connector pattern:
proto definitions + regenerated stubs, RedisProcessor (redis-py),
RedisSourceManager, facade registration, credential mapping, connector-key
maps (only host required), frontend constants, two playbook templates
(latency spike, memory & eviction pressure) and unit tests.

Fixes DrDroidLab#480

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@sidPhoenix17

Copy link
Copy Markdown
Contributor

Thanks for raising this, will take a look!

@sidPhoenix17
sidPhoenix17 self-requested a review September 10, 2026 20:26
@sidPhoenix17

Copy link
Copy Markdown
Contributor

Review: Redis connector

Overall the structure follows the existing connector pattern well (source manager + processor + protos + templates + tests). The main concern is the "Run a read-only Redis command" task: the allow/deny list approach has several holes that let a playbook author read the Redis password or mutate/block the server. Details below, ordered by severity.

🔴 Security

1. CONFIG GET leaks the Redis passwordexecutor/source_processors/redis_processor.py
CONFIG is on REDIS_READ_ONLY_COMMANDS and only SET/RESETSTAT/REWRITE are blocked. CONFIG GET requirepass (or masterauth, or CONFIG GET *) passes validation and Redis returns the password in plaintext. The result is then stored in PlayBookTaskExecutionLog.playbook_task_result and shown in the UI table, which defeats the REDIS_PASSWORD masking added in connectors/models.py. Suggest dropping CONFIG from the allow-list entirely, or allowing only CONFIG GET with a deny-list of sensitive keys (requirepass, masterauth, masteruser, user, tls-*, aclfile, dir, dbfilename).

2. Deny-list of sub-commands is structurally insufficient — same file
Verified against a local Redis 7.2 through RedisProcessor.run_command:

  • DEBUG POPULATE 1000 writes 1000 keys (DBSIZE 0 → 1000). DEBUG RELOAD forces an RDB save+reload and blocks the server. DEBUG RESTART / DEBUG CRASH-AND-RECOVER restart the process. Only DEBUG SLEEP and a handful of others are blocked. DEBUG should not be allow-listed at all (it is always enabled on Redis < 7).
  • WAIT 1 0 blocks the connection until the 15s socket timeout; WAIT is not read-only in any useful sense here.
  • CLIENT REPLY OFF succeeded and then every subsequent call on the cached self.client raised TimeoutError, so one playbook run breaks the connector for the rest of the process lifetime. CLIENT UNBLOCK <id> ERROR aborts other clients' blocking calls, CLIENT TRACKING ON, CLIENT PAUSE (blocked) etc.
  • KEYS * on a large keyspace blocks the Redis event loop; consider forcing SCAN or at least documenting it.

Recommendation: flip the model for compound commands to an allow-list of (command, subcommand) pairs (e.g. CLIENT LIST|INFO|ID|GETNAME, MEMORY USAGE|STATS|DOCTOR, SLOWLOG GET|LEN, LATENCY LATEST|HISTORY|DOCTOR, OBJECT ENCODING|FREQ|IDLETIME|REFCOUNT, XINFO STREAM|GROUPS|CONSUMERS), and remove DEBUG, WAIT, CONFIG (see #1) from the top-level list.

3. Command tokenisation uses str.split()
GET "my key" becomes three tokens. Not a security issue but worth shlex.split for correctness; also the field is MULTILINE_FT while only a single command is supported.

🟠 Correctness / completeness

4. Missing Django migrationconnectors/migrations/
Connector.connector_type, ConnectorKey.key_type, and ConnectorMetadataModelStore.connector_type/model_type use choices=generate_choices(...), so adding REDIS = 66, REDIS_* = 111-115 and REDIS_CONNECTION = 2901 needs a 0034_alter_connector_connector_type_and_more.py (same as ArgoCD/Jira did in 0033). Please run python manage.py makemigrations connectors.

5. Key-combination enumeration is incompleteconnectors/models.py
Validation is an exact set match (connectors_crud.py:158, views.py:213, connectors_update_processor.py:71, playbook_source_manager.py:123). Only 6 of the possible subsets containing REDIS_HOST are listed, so e.g. [HOST, PORT, SSL_ENABLED], [HOST, PASSWORD] or [HOST, PORT, PASSWORD, SSL_ENABLED] are rejected for API callers. The UI always submits all five keys (empty strings), so UI users are fine, but API users will hit "Missing Required Connector Keys". Either enumerate all subsets or, better, list just the full 5-key set and rely on the UI's empty-string behaviour, since RedisProcessor.__init__ already tolerates blanks.

6. RedisProcessor.get_connection caches a broken client
self.client is never reset on error. Combined with #2 (CLIENT REPLY OFF) or a network blip, subsequent tasks reuse a dead connection. Consider not caching, or resetting on exception.

7. Redis logo never usedweb/src/utils/common/cardsData.ts
web/public/integrations/redis_logo.svg is added but the REDIS card entry has no url, so TaskTitle.tsx and handleToolLogos.ts (which read cardsData.find(...).url) render nothing for Redis tasks. Add url: "/integrations/redis_logo.svg".

🟡 Minor

  • except Exception as e: raise Exception(f"Error while executing Redis task: {e}") in all three executors wraps RedisCommandNotAllowed and loses the type; the facade only needs the message so this is fine, but the per-method try/except + raise e in the processor is redundant with the logging in the manager.
  • get_slowlog re-runs int(limit) that the manager already normalised; _decode is unnecessary because decode_responses=True is set on the client.
  • ssl_enabled string comparison: str(ssl_enabled).lower() == 'true' or ssl_enabled is True — the second clause is already covered by the first.
  • redis==4.6.0 is fairly old (5.x has been stable for a while); not blocking.
  • Tests cover the happy path and a couple of blocked commands, but not CONFIG GET, DEBUG, or CLIENT cases; please add the negative cases once the allow-list is reworked.

Happy to re-review once the command guard is tightened.

- CONFIG GET could leak the Redis password (requirepass, masterauth, ...).
  CONFIG is no longer allow-listed wholesale; only CONFIG GET is permitted,
  and its result is filtered to strip sensitive keys regardless of the
  request pattern (so `CONFIG GET *` can't exfiltrate them either).
- DEBUG and WAIT removed entirely: DEBUG POPULATE writes keys, DEBUG RELOAD/
  RESTART/CRASH-AND-RECOVER disrupt the server, and WAIT blocks the
  connection - none of that belongs on a "read-only" allow-list.
- CLIENT, MEMORY, SLOWLOG, LATENCY, OBJECT, XINFO, COMMAND are no longer
  allow-listed wholesale either. Only their genuinely read-only sub-commands
  are permitted (CLIENT LIST/INFO/ID/GETNAME, MEMORY USAGE/STATS/DOCTOR,
  SLOWLOG GET/LEN, LATENCY LATEST/HISTORY/DOCTOR, OBJECT ENCODING/FREQ/
  IDLETIME/REFCOUNT, XINFO STREAM/GROUPS/CONSUMERS, COMMAND COUNT/DOCS/INFO/
  LIST/GETKEYS). CLIENT REPLY OFF in particular would also have wedged the
  connector's cached connection for every later call.
- get_connection()'s cached client is now reset on any failure instead of
  being reused, so a blocked/broken connection doesn't poison later tasks.
- Command parsing uses shlex.split() instead of str.split(), so a quoted
  argument (`GET "my key"`) is tokenised correctly.
- Reverted the Redis connector-key map to a single full 5-key set (matching
  Postgres/Clickhouse) instead of enumerating subsets - RedisProcessor
  already tolerates blank port/password/db/ssl, and the UI always submits
  all five keys anyway.
- Added the migration for the new Source/SourceKeyType/SourceModelType
  choices (0034_alter_connector_connector_type_and_more.py).
- Added negative test coverage for all of the above; verified live against
  a real Redis 7 (with requirepass set) that CONFIG GET can no longer
  surface the password and that DEBUG/WAIT/CLIENT REPLY/CLIENT KILL/MEMORY
  PURGE/SLOWLOG RESET/LATENCY RESET are all rejected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pushtisonawala

Copy link
Copy Markdown
Author

@sidPhoenix17 @dimittal
Thanks for the thorough review ,Pushed a fix:

CONFIG is now allow-list-only (GET), and the result is filtered so requirepass/masterauth/etc. can never come back, even via CONFIG GET *.
DEBUG and WAIT are removed entirely. CLIENT/MEMORY/SLOWLOG/LATENCY/OBJECT/XINFO/COMMAND are now restricted to an explicit allow-list of read-only sub-commands , verified that DEBUG SLEEP, WAIT, CLIENT REPLY OFF, CLIENT KILL, MEMORY PURGE, SLOWLOG RESET, and LATENCY RESET are all rejected against a real Redis.
Added the migration and switched tokenising to shlex.split.
Reverted the connector-key map to the single full-5-key set per your suggestion on #5
Added negative tests for all of the above (41 passing total).
Re: #7 I'm seeing url: "/integrations/redis_logo.svg" already on the REDIS entry in cardsData.ts on this branch ,let me know if you're still seeing it not render and I'll dig further.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Running redis/memcache commands

3 participants