fix(hooks): session-naming abort on role-resolution failure, not silent downgrade - #300
Merged
Brian Krabach (bkrabach) merged 1 commit intoAug 11, 2026
Conversation
…nt downgrade SessionNamingHook._call_provider silently fell back to the session's primary (expensive) model whenever a configured model_role failed to resolve. This collapsed two distinct scenarios into one silent path: 1. No model_role_resolver capability (no routing bundle installed) — legitimate fallback, correctly done 2. Resolver present but resolution fails (empty result or raised exception) — configuration failure that should not silently route to the primary model Case 2 previously logged warning but then fell through to: provider = next(iter(providers.values()), None) This caused observed production cost leak: transient resolution failures (e.g. resolver's list_models() hitting APIConnectionError) routed background naming onto claude-sonnet-5 instead of the fallback 'fast' role. Fix: Case 2 now aborts the naming attempt (return None) and logs explicit WARNING naming the consequence — naming skipped for this turn to avoid silently routing to primary provider; will retry on next turn. The resolver resolve() call is wrapped in try/except; raised exceptions treated same as empty results (previously propagated uncaught). Aborts are safe: naming is best-effort and self-retrying via the 'if current_turn >= initial_trigger_turn and not has_name:' gate. Skipping one naming attempt is zero cost; silently using the primary model costs real money and latency. Tests: 4 new tests in TestModelRoleResolution verify abort behavior (resolver empty result, resolver exception) with provider.complete() never called, plus caplog assertions for WARNING paths. Regression guards for preserved fallback (no-resolver case) and happy path already existed and pass unmodified. 21 baseline → 25 passing. Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Salil Das (sadlilas)
pushed a commit
that referenced
this pull request
Aug 18, 2026
…olution empty, not silent skip (#305) When hooks-session-naming resolves a model_role and gets a clean empty result (zero candidates), the code previously returned None and skipped session naming entirely. The feature then silently never ran, making the system harder to reason about and harder to test. A deterministic config gap should fail loud, not silent. This change falls back to the session's normal priority provider when role resolution returns cleanly empty, logs a warning naming both the role that failed to resolve and the provider actually substituted, then fires naming once per session (subsequent calls drop to DEBUG). This deliberately narrows PR #300 (fix(hooks): session-naming abort on role-resolution failure, not silent downgrade) and does not undo it: - Resolver raises an exception → still aborts. Unchanged. An exception is an unknown, possibly transient failure; falling back could route onto the expensive model on every retry — the cost leak #300 guarded against. - Resolver returns cleanly empty → now falls back loudly. This is a deterministic config gap. Aborting means the feature never runs; falling back means it at least tries and admits what it did. Related: This silent skip confounded a platform A/B test. The Linux run showed zero errors not because the defect was absent, but because naming had silently skipped and never exercised the provider code path. The Windows run had a fast candidate, ran naming, and surfaced a real provider-lifecycle defect (PR #93). Out of scope: If resolved is non-empty but no entry in providers matches the resolved name, the loop falls through to the generic fallback with no warning at all — a related silent gap left for follow-up. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SessionNamingHook._call_providersilently fell back to the session's primary (expensive) model whenever a configuredmodel_rolefailed to resolve.The code collapsed two distinct scenarios into one silent fallback:
model_role_resolvercapability registered (no routing bundle installed) — falling back to the priority provider is legitimate and intendedIn case 2, the code logged
logger.warning("model_role %r resolved to no candidates", ...)and then fell through toprovider = next(iter(providers.values()), None), quietly running a trivial background classification chore on the user's primary model.Production observation: A transient
anthropic.APIConnectionErrorinside the resolver'slist_models()made thefastrole resolve to nothing, and session naming silently ran onclaude-sonnet-5instead of haiku-4.5. Cost leak and latency regression on background work.Solution
Case 1 (no resolver capability) is unchanged — still
logger.debug+ fallback.Case 2 (resolver fails) now aborts the naming attempt (
return None) and logs aWARNINGthat names the consequence:The
await resolver.resolve(...)call is now wrapped intry/except Exception; a raised exception is treated the same as "resolved to nothing" (previously it propagated uncaught out of_call_provider).Why aborting is safe: Naming is best-effort and self-retrying: the trigger is
if current_turn >= initial_trigger_turn and not has_name:, so while the session still has no name the next turn re-attempts. The caller already treats a falsy return as a normal no-op. Skipping one naming attempt costs nothing; silently buying the primary model costs real money and latency on every attempt.No retry loops, backoff, or caching were added — this is a fail-loud fix, not a resilience redesign. (Model-list caching was addressed separately in amplifier-bundle-routing-matrix PR #41, which narrows the window; this closes the remaining case where the very first resolution of a session fails.)
Evidence
Behavioral proof with a real
SessionNamingHookand a realAnthropicProviderinstance, spying onprovider.complete()— the load-bearing question is not what got logged but whether the expensive primary model actually gets called:[](the production failure)ConnectionErrorTests:
modules/hooks-session-naming/tests/21 passing at dea5bd8 → 25 with 4 new tests inTestModelRoleResolution:test_resolver_empty_result_aborts_without_calling_provider— resolver returns[]→_call_providerreturnsNoneandprovider.completeis never awaited (load-bearing)test_resolver_exception_aborts_without_calling_provider— resolver raises → same, exception does not propagatetest_resolver_empty_result_logs_warning— caplog assertion for abort pathtest_resolver_exception_logs_warning— caplog assertion for exception pathThe two regression guards for the preserved-fallback and happy-path cases already existed in the file and pass unmodified:
test_model_role_falls_back_when_no_resolver_capabilitytest_model_role_uses_resolved_provider_and_modelNote for reviewers on running the tests: This module's tests need the compiled
amplifier_core, so they run via the amplifier tool venv's site-packages on PYTHONPATH rather than a barepytestin the module venv. See the test file header for the invocation.