Skip to content

[BE] fix: register deepseek as a canonical provider so DeepSeek model prices load#7434

Merged
andrescrz merged 1 commit into
comet-ml:mainfrom
Anuj7411:Anuj7411/deepseek-provider-registration
Jul 13, 2026
Merged

[BE] fix: register deepseek as a canonical provider so DeepSeek model prices load#7434
andrescrz merged 1 commit into
comet-ml:mainfrom
Anuj7411:Anuj7411/deepseek-provider-registration

Conversation

@Anuj7411

Copy link
Copy Markdown
Contributor

Details

Following the pattern of #7262 (which registered azure), #7432 (perplexity) and #7433 (xai) — deepseek is not currently in PROVIDERS_MAPPING, so all 12 entries in model_prices_and_context_window.json tagged with litellm_provider: "deepseek" silently drop at load time. Every one publishes non-zero input+output rates: deepseek-chat, deepseek-reasoner, deepseek-v4-flash, deepseek-v4-pro, plus the deepseek/deepseek-coder, deepseek/deepseek-r1, deepseek/deepseek-v3, deepseek/deepseek-v3.2 variants.

9 of the 12 also publish cache_read_input_token_cost. DeepSeek's cost calculator in LiteLLM (litellm/llms/deepseek/cost_calculator.py) is a one-liner that delegates directly to generic_cost_per_token — the same helper OpenAI/xAI use — so DeepSeek's usage payload follows the same OpenAI shape (prompt_tokens_details.cached_tokens flattened under original_usage.*). Route it through the existing SpanCostCalculator::textGenerationWithCacheCostOpenAI calculator; no new cache-calc method needed, just a mirror of the openai / azure / xai entries in PROVIDERS_CACHE_COST_CALCULATOR.

Change checklist

  • User facing
  • Documentation update

Issues

  • Resolves #
  • OPIK-

AI-WATERMARK

AI-WATERMARK: yes

Testing

Local verification on JDK 25 + Maven 3.9.9. Result: 102/102 green (2 new parameterized cases under calculateCostHandlesDeepseekModels, style matching #7262).

Scenarios covered:

  1. Plain text-generation routedeepseek/deepseek-coder (input 1.4e-7, output 2.8e-7, no cache rates). With prompt_tokens=1000, completion_tokens=200, expected 0.000196 (1000 * 1.4e-7 + 200 * 2.8e-7).
  2. Cache-aware route via OpenAI calcdeepseek/deepseek-chat (input 2.8e-7, output 4.2e-7, cache_read 2.8e-8). With original_usage.prompt_tokens=1000, original_usage.completion_tokens=200, original_usage.prompt_tokens_details.cached_tokens=300, expected 0.0002884 (non-cached input 700 * 2.8e-7 + 200 * 4.2e-7 + 300 * 2.8e-8).

Documentation

No docs change. Behavior change is invisible to end users beyond accurate (non-zero) cost numbers for DeepSeek traces on 12 previously-dropped models.

@Anuj7411 Anuj7411 requested a review from a team as a code owner July 11, 2026 06:24
@github-actions github-actions Bot added 🟢 size/S java Pull requests that update Java code Backend tests Including test files, or tests related like configuration. labels Jul 11, 2026
Anuj7411 added a commit to Anuj7411/opik that referenced this pull request Jul 11, 2026
…ting misses

Addressing baz-reviewer's High-severity finding on comet-ml#7432. Registering
`perplexity` as a canonical provider (this PR's original scope) fixes
direct callers, but BudgetGuard also invokes CostService.calculateCost via
LlmProviderFactoryImpl.getResolvedModelInfo(), which enumerates
`perplexity/*`, `xai/*`, and `deepseek/*` under OpenRouter and therefore
returns `provider="openrouter"`. Without an openrouter row for these
models the lookup lands on DEFAULT_COST and Perplexity-hosted evaluations
charge $0.

Add a final fallback in `findModelPrice`: when every primary lookup has
missed and the original model name carries a provider prefix that maps to
a canonical provider we know pricing for (e.g. `perplexity/sonar` maps
prefix `perplexity` -> canonical `perplexity`), retry the lookup under
that prefix's canonical provider. Only kicks in as a fallback so no
existing lookup semantics change for callers that already pass a matching
provider directly, and only when the prefix appears in PROVIDERS_MAPPING
so unmapped prefixes cannot leak through.

The same fallback covers the parallel deepseek and xai cases (comet-ml#7434,
comet-ml#7433) — those models are also enumerated under OpenRouter and would
otherwise silently drop to $0 through the aggregator-routed BudgetGuard
path.

Parameterized test uses perplexity/sonar and perplexity/sonar-pro through
both `openrouter` and `custom-llm` providers; each returns the same cost
as the direct perplexity lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@andrescrz

Copy link
Copy Markdown
Member

Hi @Anuj7411

Thanks again for the contribution!

Could you please resolve the merge conflicts in the PR? Once that's done, we'll continue with the review.

Thanks!

… prices load

Following the pattern of comet-ml#7262 (azure), comet-ml#7432 (perplexity) and comet-ml#7433
(xai) — `deepseek` is not currently in PROVIDERS_MAPPING, so all 12
entries in model_prices_and_context_window.json tagged with
litellm_provider: "deepseek" silently drop at load time. Every one
publishes non-zero input+output rates: deepseek-chat, deepseek-reasoner,
deepseek-v4-flash, deepseek-v4-pro, plus the deepseek/deepseek-coder,
deepseek/deepseek-r1, deepseek/deepseek-v3, deepseek/deepseek-v3.2
variants.

9 of the 12 also publish cache_read_input_token_cost. DeepSeek's cost
calc in LiteLLM (litellm/llms/deepseek/cost_calculator.py) is a
one-liner that delegates directly to generic_cost_per_token — the same
helper OpenAI/xAI use — so DeepSeek's usage payload follows OpenAI's
shape (prompt_tokens_details.cached_tokens flattened under
original_usage.*). Route it through the existing
textGenerationWithCacheCostOpenAI calculator; no new cache-calc method
needed.

Parameterized test uses deepseek/deepseek-coder (no cache ->
textGenerationCost) and deepseek/deepseek-chat (input 2.8e-7, output
4.2e-7, cache_read 2.8e-8 -> textGenerationWithCacheCostOpenAI). Both
branches exercised in regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Anuj7411 Anuj7411 force-pushed the Anuj7411/deepseek-provider-registration branch from d38f001 to 43535ef Compare July 13, 2026 10:24
Anuj7411 added a commit to Anuj7411/opik that referenced this pull request Jul 13, 2026
…ting misses

Addressing baz-reviewer's High-severity finding on comet-ml#7432. Registering
`perplexity` as a canonical provider (this PR's original scope) fixes
direct callers, but BudgetGuard also invokes CostService.calculateCost via
LlmProviderFactoryImpl.getResolvedModelInfo(), which enumerates
`perplexity/*`, `xai/*`, and `deepseek/*` under OpenRouter and therefore
returns `provider="openrouter"`. Without an openrouter row for these
models the lookup lands on DEFAULT_COST and Perplexity-hosted evaluations
charge $0.

Add a final fallback in `findModelPrice`: when every primary lookup has
missed and the original model name carries a provider prefix that maps to
a canonical provider we know pricing for (e.g. `perplexity/sonar` maps
prefix `perplexity` -> canonical `perplexity`), retry the lookup under
that prefix's canonical provider. Only kicks in as a fallback so no
existing lookup semantics change for callers that already pass a matching
provider directly, and only when the prefix appears in PROVIDERS_MAPPING
so unmapped prefixes cannot leak through.

The same fallback covers the parallel deepseek and xai cases (comet-ml#7434,
comet-ml#7433) — those models are also enumerated under OpenRouter and would
otherwise silently drop to $0 through the aggregator-routed BudgetGuard
path.

Parameterized test uses perplexity/sonar and perplexity/sonar-pro through
both `openrouter` and `custom-llm` providers; each returns the same cost
as the direct perplexity lookup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Anuj7411

Copy link
Copy Markdown
Contributor Author

Rebased on latest main; conflicts were purely additive (xai landed first, so this now slots deepseek alongside xai in PROVIDERS_MAPPING and PROVIDERS_CACHE_COST_CALCULATOR, plus the two new test methods sit side-by-side). No logic changes on either commit. Ready for re-review

@andrescrz andrescrz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Anuj7411

Thanks for the contribution! I reviewed the PR and everything looks good from my side. I've approved it, and all CI checks have passed successfully.

We'll merge it shortly, and the change will be included in the next Opik release.

Thanks again for contributing to Opik!

@andrescrz andrescrz merged commit 76ce027 into comet-ml:main Jul 13, 2026
33 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend java Pull requests that update Java code 🟢 size/S tests Including test files, or tests related like configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants