[BE] fix: register deepseek as a canonical provider so DeepSeek model prices load#7434
Conversation
…ting misses Addressing baz-reviewer's High-severity finding on comet-ml#7432. Registering `perplexity` as a canonical provider (this PR's original scope) fixes direct callers, but BudgetGuard also invokes CostService.calculateCost via LlmProviderFactoryImpl.getResolvedModelInfo(), which enumerates `perplexity/*`, `xai/*`, and `deepseek/*` under OpenRouter and therefore returns `provider="openrouter"`. Without an openrouter row for these models the lookup lands on DEFAULT_COST and Perplexity-hosted evaluations charge $0. Add a final fallback in `findModelPrice`: when every primary lookup has missed and the original model name carries a provider prefix that maps to a canonical provider we know pricing for (e.g. `perplexity/sonar` maps prefix `perplexity` -> canonical `perplexity`), retry the lookup under that prefix's canonical provider. Only kicks in as a fallback so no existing lookup semantics change for callers that already pass a matching provider directly, and only when the prefix appears in PROVIDERS_MAPPING so unmapped prefixes cannot leak through. The same fallback covers the parallel deepseek and xai cases (comet-ml#7434, comet-ml#7433) — those models are also enumerated under OpenRouter and would otherwise silently drop to $0 through the aggregator-routed BudgetGuard path. Parameterized test uses perplexity/sonar and perplexity/sonar-pro through both `openrouter` and `custom-llm` providers; each returns the same cost as the direct perplexity lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Hi @Anuj7411 Thanks again for the contribution! Could you please resolve the merge conflicts in the PR? Once that's done, we'll continue with the review. Thanks! |
… prices load Following the pattern of comet-ml#7262 (azure), comet-ml#7432 (perplexity) and comet-ml#7433 (xai) — `deepseek` is not currently in PROVIDERS_MAPPING, so all 12 entries in model_prices_and_context_window.json tagged with litellm_provider: "deepseek" silently drop at load time. Every one publishes non-zero input+output rates: deepseek-chat, deepseek-reasoner, deepseek-v4-flash, deepseek-v4-pro, plus the deepseek/deepseek-coder, deepseek/deepseek-r1, deepseek/deepseek-v3, deepseek/deepseek-v3.2 variants. 9 of the 12 also publish cache_read_input_token_cost. DeepSeek's cost calc in LiteLLM (litellm/llms/deepseek/cost_calculator.py) is a one-liner that delegates directly to generic_cost_per_token — the same helper OpenAI/xAI use — so DeepSeek's usage payload follows OpenAI's shape (prompt_tokens_details.cached_tokens flattened under original_usage.*). Route it through the existing textGenerationWithCacheCostOpenAI calculator; no new cache-calc method needed. Parameterized test uses deepseek/deepseek-coder (no cache -> textGenerationCost) and deepseek/deepseek-chat (input 2.8e-7, output 4.2e-7, cache_read 2.8e-8 -> textGenerationWithCacheCostOpenAI). Both branches exercised in regressions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
d38f001 to
43535ef
Compare
…ting misses Addressing baz-reviewer's High-severity finding on comet-ml#7432. Registering `perplexity` as a canonical provider (this PR's original scope) fixes direct callers, but BudgetGuard also invokes CostService.calculateCost via LlmProviderFactoryImpl.getResolvedModelInfo(), which enumerates `perplexity/*`, `xai/*`, and `deepseek/*` under OpenRouter and therefore returns `provider="openrouter"`. Without an openrouter row for these models the lookup lands on DEFAULT_COST and Perplexity-hosted evaluations charge $0. Add a final fallback in `findModelPrice`: when every primary lookup has missed and the original model name carries a provider prefix that maps to a canonical provider we know pricing for (e.g. `perplexity/sonar` maps prefix `perplexity` -> canonical `perplexity`), retry the lookup under that prefix's canonical provider. Only kicks in as a fallback so no existing lookup semantics change for callers that already pass a matching provider directly, and only when the prefix appears in PROVIDERS_MAPPING so unmapped prefixes cannot leak through. The same fallback covers the parallel deepseek and xai cases (comet-ml#7434, comet-ml#7433) — those models are also enumerated under OpenRouter and would otherwise silently drop to $0 through the aggregator-routed BudgetGuard path. Parameterized test uses perplexity/sonar and perplexity/sonar-pro through both `openrouter` and `custom-llm` providers; each returns the same cost as the direct perplexity lookup. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Rebased on latest main; conflicts were purely additive (xai landed first, so this now slots deepseek alongside xai in PROVIDERS_MAPPING and PROVIDERS_CACHE_COST_CALCULATOR, plus the two new test methods sit side-by-side). No logic changes on either commit. Ready for re-review |
andrescrz
left a comment
There was a problem hiding this comment.
Hi @Anuj7411
Thanks for the contribution! I reviewed the PR and everything looks good from my side. I've approved it, and all CI checks have passed successfully.
We'll merge it shortly, and the change will be included in the next Opik release.
Thanks again for contributing to Opik!
Details
Following the pattern of #7262 (which registered
azure), #7432 (perplexity) and #7433 (xai) —deepseekis not currently inPROVIDERS_MAPPING, so all 12 entries inmodel_prices_and_context_window.jsontagged withlitellm_provider: "deepseek"silently drop at load time. Every one publishes non-zero input+output rates:deepseek-chat,deepseek-reasoner,deepseek-v4-flash,deepseek-v4-pro, plus thedeepseek/deepseek-coder,deepseek/deepseek-r1,deepseek/deepseek-v3,deepseek/deepseek-v3.2variants.9 of the 12 also publish
cache_read_input_token_cost. DeepSeek's cost calculator in LiteLLM (litellm/llms/deepseek/cost_calculator.py) is a one-liner that delegates directly togeneric_cost_per_token— the same helper OpenAI/xAI use — so DeepSeek's usage payload follows the same OpenAI shape (prompt_tokens_details.cached_tokensflattened underoriginal_usage.*). Route it through the existingSpanCostCalculator::textGenerationWithCacheCostOpenAIcalculator; no new cache-calc method needed, just a mirror of theopenai/azure/xaientries inPROVIDERS_CACHE_COST_CALCULATOR.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
Local verification on JDK 25 + Maven 3.9.9. Result: 102/102 green (2 new parameterized cases under
calculateCostHandlesDeepseekModels, style matching #7262).Scenarios covered:
deepseek/deepseek-coder(input1.4e-7, output2.8e-7, no cache rates). Withprompt_tokens=1000, completion_tokens=200, expected0.000196(1000 * 1.4e-7 + 200 * 2.8e-7).deepseek/deepseek-chat(input2.8e-7, output4.2e-7,cache_read 2.8e-8). Withoriginal_usage.prompt_tokens=1000, original_usage.completion_tokens=200, original_usage.prompt_tokens_details.cached_tokens=300, expected0.0002884(non-cached input700 * 2.8e-7 + 200 * 4.2e-7 + 300 * 2.8e-8).Documentation
No docs change. Behavior change is invisible to end users beyond accurate (non-zero) cost numbers for DeepSeek traces on 12 previously-dropped models.