Repository navigation
Conversation
Collaborator
|
动机和方案并不认可。以及未实机验证 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
关联 #266(“OCR 功能依赖 GPU,能否用多模态模型做识别”),为其中“用多模态模型做识别”的诉求提供一个可直接选择的引擎。
本 PR 新增
minicpm_ocr引擎:Yuxi 只把 PDF 页面/图片发送给用户自托管的 vLLM(MiniCPM-OCR 1.0 的 OpenAI 兼容服务),不在 API 镜像内加载模型,也不需要本机 GPU。定位与现有deepseek_ocr相同——远端 VLM 识别,逐页返回文本。改动
backend/yuxi/infrastructure/document_parsing/engines/minicpm_ocr.py:新引擎,复用DocumentEngine契约、PDFIUM_LOCK与render_pdf_page_image,PDF 逐页渲染后识别,图片直接读取。engines/__init__.py:注册minicpm_ocr(支持.pdf/.png/.jpg/.jpeg/.bmp/.webp,与deepseek_ocr一致)。modules/documents/options.py:新增minicpm_ocr_opts(api_url、可选api_key、model),环境变量MINICPM_OCR_API_URL/MINICPM_OCR_API_KEY。前端配置页由服务端字段驱动,无需改动。docs/advanced/document-processing.md增加引擎说明与 vLLM 启动命令;新增 decision record。协议细节
官方 vLLM 部署要求每个请求携带多模态切片参数,缺少时服务端拒绝或降质,因此请求固定包含:
{ "mm_processor_kwargs": {"downsample_mode": "4x", "max_slice_nums": 9}, "chat_template_kwargs": {"enable_thinking": false}, "temperature": 0.01, "top_p": 0.001, "top_k": 1, "repetition_penalty": 1.0, "no_repeat_ngram_size": 100 }image_url→Text Recognition:,与官方示例一致。Authorization,兼容未启用--api-key的部署。finish_reason != "stop"或正文不是字符串时失败,沿用 parser 既有发布边界,不产生document.md。/v1/models。验证
pytest test/unit/knowledge/test_minicpm_ocr.py test/unit/knowledge/test_parser_capabilities.py -q→ 20 passedpytest test/unit/knowledge -q -m "not slow"→ 534 passed, 1 skippedpytest test/unit/services/test_ocr_service.py -q→ 15 passed(包含新增的MINICPM_OCR_API_URL解析用例)ruff check(改动文件)→ 通过;python3 scripts/ci/verify_engineering_contracts.py→ 通过test/integration/api/test_system_router_api.py需要真实 Compose 栈,本机 skipped,未计入通过;已同步更新其option_keys断言。未验证:本机没有 GPU / vLLM 服务,未做真实 MiniCPM-OCR 调用,协议层回放不代表识别质量。另外本机
test/unit/services的 5 个 skills/workspace 用例与test/unit/config的 docker compose 用例失败,stash 本次改动后基线同样失败,与本次改动无关。取舍
minicpm-ocrpip SDK(PP-DocLayoutV3 版面检测),因为它会带来 torch/paddle 级依赖与模型下载,和“不要求本机 GPU”的目标冲突。