Skip to content

SearchIndex.from_existing silently drops vector fields on a RESP2 connection with legacy_responses=False #712

Description

@vishal-bala

Round-tripping an index through SearchIndex.from_existing and then create() rebuilds it without its vector field, silently, when the supplied client is a RESP2 connection created with redis-py 8's legacy_responses=False. Nothing is logged and nothing is raised. For a vector search library this is the worst available failure mode: the rebuilt index reports itself as healthy, FT.INFO lists it as built, and every vector query against it is searching for a field that no longer exists.

Reproduction

Measured on Redis 8.4.6 with redis-py 8.1.0, RedisVL at edcc78e. The source index has three fields, one of them a FLAT vector field named emb.

from redis import Redis
from redisvl.index import SearchIndex

client = Redis.from_url("redis://localhost:6379", protocol=2, legacy_responses=False)
recovered = SearchIndex.from_existing("my_index", redis_client=client)
print(sorted(recovered.schema.field_names))   # ['cat', 'text'], and 'emb' is gone

recovered.schema.index.name = "my_index_rebuilt"
recovered.create(overwrite=True)

FT.INFO my_index_rebuilt then reports attributes [b'text', b'cat']. The rebuilt index really does lack the vector field; that is a measured result rather than an inference from the schema.

Cause

redis-py's two unified FT.INFO parsers disagree about where vector parameters live. On a RESP3 connection they are promoted to top-level keys, measured on the same index as:

{b'identifier': b'emb', b'attribute': b'emb', b'type': b'VECTOR',
 b'algorithm': b'FLAT', b'data_type': b'FLOAT32', b'dim': 4,
 b'distance_metric': b'COSINE', b'flags': []}

On a RESP2 connection with legacy_responses=False, _parse_info_unified packs the same parameters into the flags list instead:

{'identifier': 'emb', 'attribute': 'emb', 'type': 'VECTOR',
 'flags': ['algorithm', 'FLAT', 'data_type', 'FLOAT32', 'dim', 4,
           'distance_metric', 'COSINE']}

parse_vector_attrs in redisvl/redis/connection.py takes its dict branch here and excludes flags from the keys it copies (line 321), so vector_attrs comes out empty. dims is therefore absent, and because dims is required the function returns None. The caller at line 548 reads that None as "vector attributes cannot be parsed on this Redis version" and does a bare continue at line 551, dropping the field from the schema without a word.

The continue is defensible on its own terms, since it was written for Redis 6.2.6-v9 where FT.INFO genuinely returns no vector parameters. What is not defensible is that it now also absorbs a parse failure on a modern server where the parameters were present in the reply all along.

Scope

Measured across three client configurations against the same index:

Client Fields recovered by from_existing
protocol=2 (RedisVL's own default) ['emb', 'text']
protocol=2, legacy_responses=False ['text']
protocol=3 ['emb', 'text']

Only the middle row is affected, so a fix should target the RESP2 unified shape rather than RESP3 handling, which already works. Note that legacy_responses is a redis-py 8 only keyword; on 6.3.0 and 7.4.0 it raises TypeError from AbstractConnection.__init__ at connect time, so this configuration is reachable only on redis-py 8.

Suggested fix

Two changes, independently useful. First, teach the dict branch of parse_vector_attrs to unpack a non-empty flags list as alternating key-value pairs before excluding it, which restores dims and the rest of the parameters. Second, replace the bare continue with a warning naming the field and the index, so that a future parse gap degrades loudly instead of silently. The second is worth doing regardless of the first: any path that yields a schema missing a field the server reported should say so.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions