Repository navigation
fix(models): 校验 Embedding 响应与输入逐条对应 - #1143
Merged
Merged
Conversation
_extract_embeddings 只检查响应里有没有 data,就把 data[*].embedding 拼成列表; batch_encode / abatch_encode 逐批 extend 时也不校验该批条数。 MilvusKB._insert_chunks_to_stores 按列组装 entities,向量列与 chunk 列靠位置 对应,而 pymilvus 只校验列数、不校验各列长度一致(实测 [3,3,3,2] 可被接受)。 Embedding 端点返回部分结果时,缺向量的 chunk 与错位的向量会静默写入 Milvus, 而 PostgreSQL 的 chunk 事实已先行写入。 按 rerank 在 xerrors#1016 建立的同类契约处理:响应条数必须等于请求条数,index 存在时 必须是范围内唯一整数,否则整批抛出固定错误;无 index 的既有响应保持按顺序接受。 批次累加前额外校验每批条数,避免子集被拼进更大的错位列表。 Fixes xerrors#1142
Owner
|
已提交 验证:原实现新增回归 6 失败,修复后模型模块 129 通过;Ruff、工程契约(含 56 项测试)及文档构建通过。CI 待最新提交运行,通过且无阻塞后合并。 执行命令与环境限制
|
xerrors
pushed a commit
that referenced
this pull request
Nov 4, 2026
_extract_embeddings 在 #1143 建立了「响应与输入逐条对应」的契约,但不校验每条 向量的长度;test_connection 只比对 embeddings[0],且仅用于连接探针。 MilvusKB._create_new_collection 用配置维度固定集合 schema,写入时不再比对; pymilvus 只校验列数,实测 Prepare.prepare_data 接受 [4, 2, 4] 对 schema dim=4 的批次。供应商切换模型版本、截断向量或聚合网关路由到维度不同的后端时,维度 混杂的批次会静默写入集合,而 PostgreSQL 的 chunk 事实已先行写入。 在响应边界新增维度校验:_extract_embeddings 接收 dimension,逐条比对长度; _require_one_vector_per_input 一并校验,供 batch_encode / abatch_encode 每批复用。 dimension 为 None 时保持既有行为。 Fixes #1144
This was referenced Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
修改内容
Embedding响应必须与请求输入逐条对应,否则整批失败;修改见backend/yuxi/modules/models/embed.py。_extract_embeddings原先只检查响应里有没有data,就把data[*].embedding拼成列表;batch_encode/abatch_encode逐批extend时也不校验该批条数。MilvusKB._insert_chunks_to_stores按列组装entities,向量列与chunk_id/content列靠位置对应,而pymilvus只校验列数等于 schema 字段数、不校验各列长度一致。我用 pymilvus 3.0.2 实测:因此 Embedding 端点返回部分结果时不会抛错:缺向量的 chunk 和错位的向量会静默写入 Milvus,而 PostgreSQL 的 chunk 事实已先行写入(
_insert_chunks_to_stores注释写明「PG 为权威源,先落 chunk 事实」),属于无法自动发现的静默数据损坏。与既有契约的关系
models/rerank.py的_batch_rerank已在 #1016 建立同类契约:结果条数等于输入条数、index为范围内唯一整数,否则整批抛出固定错误。本 PR 让 Embedding 遵守同一契约。决策
_extract_embeddings接收请求条数,条数不符直接抛出带条数信息的ValueError。index时按index归位;缺失、重复、越界(含超出请求条数)的索引整批失败,混用带与不带index的条目同样失败。index的既有响应继续按响应顺序接受,保持兼容。batch_encode/abatch_encode在每批extend前校验该批条数,避免子集被拼进更大的错位列表。不填零向量、不静默截断:两种做法都会让错配继续存在且不可见。
详情见
docs/develop-guides/decisions/implemented/2026-11-03-embedding-response-alignment.md。验证
backend/test/unit/models/test_embed_alignment.py,8 个用例:条数不足、重复索引、越界索引、乱序归位、无索引兼容、空输入、encode拒绝部分批次、abatch_encode逐批拒绝部分结果。ruff check、ruff format --check、ruff check --select I在backend/下对改动文件通过。未覆盖
test/unit全量套件受 POSIX 专属路径标记阻塞(os.O_DIRECTORY),因此只在隔离环境运行了本 PR 新增用例;容器内的全量test/unit需由 CI 或维护者执行。Fixes #1142