Skip to content

fix(pricing): prefer a native provider over aggregator re-listings - #1279

Open
samirhvbr wants to merge 1 commit into
usestrix:mainfrom
samirhvbr:fix/pricing-prefer-native-provider
Open

fix(pricing): prefer a native provider over aggregator re-listings#1279
samirhvbr wants to merge 1 commit into
usestrix:mainfrom
samirhvbr:fix/pricing-prefer-native-provider

Conversation

@samirhvbr

Copy link
Copy Markdown

Problem

resolve_litellm_model() resolves a bare model name that several LiteLLM providers list by returning the alphabetically first matching key when those listings agree on price. LiteLLM's model_cost has since grown aggregator re-listings that sort ahead of a model's first-party provider. With the pinned litellm==1.90.1, grok-4.5 now has three same-priced listings:

  • openrouter/x-ai/grok-4.5
  • perplexity/xai/grok-4.5
  • xai/grok-4.5

so the resolver returns openrouter/x-ai/grok-4.5 instead of xai/grok-4.5. The cost estimate is unaffected (identical price), but the reported provider is wrong — and tests/test_pricing.py::test_resolves_common_bare_model_names fails on main against the pinned litellm.

Fix

Prefer the first-party listing: the match whose key is exactly <litellm_provider>/<name>. When that match is unique, return it before falling back to price consensus. This restores native-provider attribution and also resolves names the price-consensus rule gave up on when aggregators disagree on price (a unique first-party listing among differently priced mirrors).

Tests

  • test_resolves_common_bare_model_names passes again against litellm 1.90.1.
  • Adds two deterministic, mock-backed regression tests so the behavior no longer rides on the live LiteLLM catalog.
  • ruff format, ruff check, mypy, and pytest tests/test_pricing.py all pass.

🤖 Generated with Claude Code

resolve_litellm_model() tie-breaks a bare model name that several LiteLLM
providers list by returning the alphabetically first key whenever those
listings agree on price. As LiteLLM's model_cost has grown, aggregators
that re-key a model under a namespaced path now sort ahead of the model's
own first-party provider: with litellm 1.90.1, "grok-4.5" resolves to
"openrouter/x-ai/grok-4.5" instead of "xai/grok-4.5". The estimate stays
correct (same price), but the reported provider is wrong, and it breaks
test_resolves_common_bare_model_names on the pinned litellm.

Prefer the first-party listing: the match whose key is exactly
"<litellm_provider>/<name>". When that is unique, return it before falling
back to price consensus. This restores native-provider attribution and
also resolves names the old rule gave up on when aggregators disagreed on
price (e.g. a unique first-party listing among differently priced mirrors).

Add deterministic, mock-backed regression tests for both cases so the
behavior no longer rides on live LiteLLM catalog data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 6, 2026 14:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped, improves determinism/provider attribution, and is covered by targeted regression tests.

Pull request overview

This PR fixes LiteLLM model-name resolution so that when a bare model name matches both aggregator re-listings and the model’s first-party provider listing, the resolver prefers the native provider key (e.g., xai/grok-4.5) rather than the alphabetically first aggregator key. This keeps provider attribution accurate and stabilizes resolution behavior as LiteLLM’s model_cost catalog grows.

Changes:

  • Update resolve_litellm_model() to prefer a unique first-party (“native”) "<litellm_provider>/<name>" match before falling back to the existing price-consensus tie-break.
  • Add two mock-backed regression tests covering native-vs-aggregator preference and the “unique native even when aggregators disagree on price” case.
File summaries
File Description
strix/report/pricing.py Adjusts model resolution logic to prioritize unique native provider listings over aggregator re-listings.
tests/test_pricing.py Adds deterministic tests that pin the resolver behavior for native-provider preference scenarios.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@greptile-apps

greptile-apps Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates LiteLLM model resolution to prefer a unique first-party provider listing over aggregator re-listings before applying price consensus.

  • Detects native listings using the catalog entry’s litellm_provider.
  • Preserves the existing price-consensus fallback when no unique native listing exists.
  • Adds deterministic regression coverage for equal-price and differing-price aggregator listings.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness, security, or repository-rule violations identified.

The native-provider preference is applied only when exactly one matching catalog key identifies itself as the provider’s direct listing, while ambiguous cases retain the existing price-consensus behavior.

Important Files Changed

Filename Overview
strix/report/pricing.py Adds a narrowly scoped native-provider preference while retaining existing exact-match and consensus behavior.
tests/test_pricing.py Adds isolated regression tests that restore LiteLLM’s global catalog and clear the resolver cache.

Reviews (1): Last reviewed commit: "fix(pricing): prefer a native provider o..." | Re-trigger Greptile

samirhvbr added a commit to samirhvbr/strix that referenced this pull request Sep 6, 2026
…#1279)

Bump .fork-version to 1.5.3+shvia.14 and record the LiteLLM native-provider
resolution fix (cherry-picked from the upstream PR usestrix#1279 branch) in FORK.md.
Also realign version.md's "Versão atual" line, which was stale at shvia.12.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
samirhvbr added a commit to samirhvbr/strix that referenced this pull request Sep 6, 2026
Sync the fork line onto usestrix/strix v1.6.2 (was 1.5.3). Conflict
resolution and reconciliation of the fork's patches:

- warmup: adopt upstream's wait_for_import_warmup() approach and drop the
  fork's synchronous agents pre-import (_preimport_thread_unsafe_sdk), which
  upstream's own warmup rework supersedes. Remove the now-obsolete
  tests/test_warmup.py (upstream covers this in tests/test_import_warmup.py).
- runner: keep upstream's dedicated `except RateLimitError` block; narrow the
  fork's F3 usage-limit handler to the non-RateLimitError (LiteLLM) case so
  there is no dead code, and record the same "rate_limited" exit reason.
- viewer: rebuild the compiled frontend bundle (static/) from the merged
  source so the fork's local-PDF button rides on upstream's new frontend.
- test_viewer: adapt the historical-run-data test to the fork's session-only
  gating (the fork removed the upstream email-verification gate).

Full suite: 1740 passed (isolated HOME). The pricing native-provider fix
(shvia.14, PR usestrix#1279) is carried on top.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants