fix(pricing): prefer a native provider over aggregator re-listings - #1279
fix(pricing): prefer a native provider over aggregator re-listings#1279samirhvbr wants to merge 1 commit into
Conversation
resolve_litellm_model() tie-breaks a bare model name that several LiteLLM providers list by returning the alphabetically first key whenever those listings agree on price. As LiteLLM's model_cost has grown, aggregators that re-key a model under a namespaced path now sort ahead of the model's own first-party provider: with litellm 1.90.1, "grok-4.5" resolves to "openrouter/x-ai/grok-4.5" instead of "xai/grok-4.5". The estimate stays correct (same price), but the reported provider is wrong, and it breaks test_resolves_common_bare_model_names on the pinned litellm. Prefer the first-party listing: the match whose key is exactly "<litellm_provider>/<name>". When that is unique, return it before falling back to price consensus. This restores native-provider attribution and also resolves names the old rule gave up on when aggregators disagreed on price (e.g. a unique first-party listing among differently priced mirrors). Add deterministic, mock-backed regression tests for both cases so the behavior no longer rides on live LiteLLM catalog data. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
The change is narrowly scoped, improves determinism/provider attribution, and is covered by targeted regression tests.
Pull request overview
This PR fixes LiteLLM model-name resolution so that when a bare model name matches both aggregator re-listings and the model’s first-party provider listing, the resolver prefers the native provider key (e.g., xai/grok-4.5) rather than the alphabetically first aggregator key. This keeps provider attribution accurate and stabilizes resolution behavior as LiteLLM’s model_cost catalog grows.
Changes:
- Update
resolve_litellm_model()to prefer a unique first-party (“native”)"<litellm_provider>/<name>"match before falling back to the existing price-consensus tie-break. - Add two mock-backed regression tests covering native-vs-aggregator preference and the “unique native even when aggregators disagree on price” case.
File summaries
| File | Description |
|---|---|
strix/report/pricing.py |
Adjusts model resolution logic to prioritize unique native provider listings over aggregator re-listings. |
tests/test_pricing.py |
Adds deterministic tests that pin the resolver behavior for native-provider preference scenarios. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Greptile SummaryThis PR updates LiteLLM model resolution to prefer a unique first-party provider listing over aggregator re-listings before applying price consensus.
Confidence Score: 5/5The PR appears safe to merge with no actionable correctness, security, or repository-rule violations identified. The native-provider preference is applied only when exactly one matching catalog key identifies itself as the provider’s direct listing, while ambiguous cases retain the existing price-consensus behavior. Important Files Changed
Reviews (1): Last reviewed commit: "fix(pricing): prefer a native provider o..." | Re-trigger Greptile |
…#1279) Bump .fork-version to 1.5.3+shvia.14 and record the LiteLLM native-provider resolution fix (cherry-picked from the upstream PR usestrix#1279 branch) in FORK.md. Also realign version.md's "Versão atual" line, which was stale at shvia.12. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sync the fork line onto usestrix/strix v1.6.2 (was 1.5.3). Conflict resolution and reconciliation of the fork's patches: - warmup: adopt upstream's wait_for_import_warmup() approach and drop the fork's synchronous agents pre-import (_preimport_thread_unsafe_sdk), which upstream's own warmup rework supersedes. Remove the now-obsolete tests/test_warmup.py (upstream covers this in tests/test_import_warmup.py). - runner: keep upstream's dedicated `except RateLimitError` block; narrow the fork's F3 usage-limit handler to the non-RateLimitError (LiteLLM) case so there is no dead code, and record the same "rate_limited" exit reason. - viewer: rebuild the compiled frontend bundle (static/) from the merged source so the fork's local-PDF button rides on upstream's new frontend. - test_viewer: adapt the historical-run-data test to the fork's session-only gating (the fork removed the upstream email-verification gate). Full suite: 1740 passed (isolated HOME). The pricing native-provider fix (shvia.14, PR usestrix#1279) is carried on top. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Problem
resolve_litellm_model()resolves a bare model name that several LiteLLM providers list by returning the alphabetically first matching key when those listings agree on price. LiteLLM'smodel_costhas since grown aggregator re-listings that sort ahead of a model's first-party provider. With the pinnedlitellm==1.90.1,grok-4.5now has three same-priced listings:openrouter/x-ai/grok-4.5perplexity/xai/grok-4.5xai/grok-4.5so the resolver returns
openrouter/x-ai/grok-4.5instead ofxai/grok-4.5. The cost estimate is unaffected (identical price), but the reported provider is wrong — andtests/test_pricing.py::test_resolves_common_bare_model_namesfails onmainagainst the pinned litellm.Fix
Prefer the first-party listing: the match whose key is exactly
<litellm_provider>/<name>. When that match is unique, return it before falling back to price consensus. This restores native-provider attribution and also resolves names the price-consensus rule gave up on when aggregators disagree on price (a unique first-party listing among differently priced mirrors).Tests
test_resolves_common_bare_model_namespasses again against litellm 1.90.1.ruff format,ruff check,mypy, andpytest tests/test_pricing.pyall pass.🤖 Generated with Claude Code