Skip to content

fix(hooks): session-naming abort on role-resolution failure, not silent downgrade - #300

Merged
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
fix/session-naming-abort-on-role-resolution-failure
Aug 11, 2026
Merged

fix(hooks): session-naming abort on role-resolution failure, not silent downgrade#300
Brian Krabach (bkrabach) merged 1 commit into
mainfrom
fix/session-naming-abort-on-role-resolution-failure

Conversation

@bkrabach

Copy link
Copy Markdown
Collaborator

Problem

SessionNamingHook._call_provider silently fell back to the session's primary (expensive) model whenever a configured model_role failed to resolve.

The code collapsed two distinct scenarios into one silent fallback:

  1. No model_role_resolver capability registered (no routing bundle installed) — falling back to the priority provider is legitimate and intended
  2. Resolver IS registered but resolution fails (returns no candidates, or raises) — this is a failure of an explicitly-configured routing preference

In case 2, the code logged logger.warning("model_role %r resolved to no candidates", ...) and then fell through to provider = next(iter(providers.values()), None), quietly running a trivial background classification chore on the user's primary model.

Production observation: A transient anthropic.APIConnectionError inside the resolver's list_models() made the fast role resolve to nothing, and session naming silently ran on claude-sonnet-5 instead of haiku-4.5. Cost leak and latency regression on background work.

Solution

Case 1 (no resolver capability) is unchanged — still logger.debug + fallback.

Case 2 (resolver fails) now aborts the naming attempt (return None) and logs a WARNING that names the consequence:

  • "skipping session naming for this turn to avoid silently routing onto the primary (expensive) provider"
  • "will retry on a later turn"

The await resolver.resolve(...) call is now wrapped in try/except Exception; a raised exception is treated the same as "resolved to nothing" (previously it propagated uncaught out of _call_provider).

Why aborting is safe: Naming is best-effort and self-retrying: the trigger is if current_turn >= initial_trigger_turn and not has_name:, so while the session still has no name the next turn re-attempts. The caller already treats a falsy return as a normal no-op. Skipping one naming attempt costs nothing; silently buying the primary model costs real money and latency on every attempt.

No retry loops, backoff, or caching were added — this is a fail-loud fix, not a resilience redesign. (Model-list caching was addressed separately in amplifier-bundle-routing-matrix PR #41, which narrows the window; this closes the remaining case where the very first resolution of a session fails.)

Evidence

Behavioral proof with a real SessionNamingHook and a real AnthropicProvider instance, spying on provider.complete() — the load-bearing question is not what got logged but whether the expensive primary model actually gets called:

scenario primary model called?
resolver returns [] (the production failure) False — aborted
resolver raises ConnectionError False — aborted, no propagation
no resolver capability at all (regression guard) True — legitimate fallback preserved

Tests: modules/hooks-session-naming/tests/ 21 passing at dea5bd8 → 25 with 4 new tests in TestModelRoleResolution:

  • test_resolver_empty_result_aborts_without_calling_provider — resolver returns []_call_provider returns None and provider.complete is never awaited (load-bearing)
  • test_resolver_exception_aborts_without_calling_provider — resolver raises → same, exception does not propagate
  • test_resolver_empty_result_logs_warning — caplog assertion for abort path
  • test_resolver_exception_logs_warning — caplog assertion for exception path

The two regression guards for the preserved-fallback and happy-path cases already existed in the file and pass unmodified:

  • test_model_role_falls_back_when_no_resolver_capability
  • test_model_role_uses_resolved_provider_and_model

Note for reviewers on running the tests: This module's tests need the compiled amplifier_core, so they run via the amplifier tool venv's site-packages on PYTHONPATH rather than a bare pytest in the module venv. See the test file header for the invocation.

…nt downgrade

SessionNamingHook._call_provider silently fell back to the session's primary
(expensive) model whenever a configured model_role failed to resolve. This
collapsed two distinct scenarios into one silent path:

1. No model_role_resolver capability (no routing bundle installed) — legitimate
   fallback, correctly done
2. Resolver present but resolution fails (empty result or raised exception) —
   configuration failure that should not silently route to the primary model

Case 2 previously logged warning but then fell through to:
  provider = next(iter(providers.values()), None)
This caused observed production cost leak: transient resolution failures
(e.g. resolver's list_models() hitting APIConnectionError) routed background
naming onto claude-sonnet-5 instead of the fallback 'fast' role.

Fix: Case 2 now aborts the naming attempt (return None) and logs explicit
WARNING naming the consequence — naming skipped for this turn to avoid
silently routing to primary provider; will retry on next turn. The resolver
resolve() call is wrapped in try/except; raised exceptions treated same as
empty results (previously propagated uncaught).

Aborts are safe: naming is best-effort and self-retrying via the
'if current_turn >= initial_trigger_turn and not has_name:' gate. Skipping
one naming attempt is zero cost; silently using the primary model costs real
money and latency.

Tests: 4 new tests in TestModelRoleResolution verify abort behavior (resolver
empty result, resolver exception) with provider.complete() never called, plus
caplog assertions for WARNING paths. Regression guards for preserved fallback
(no-resolver case) and happy path already existed and pass unmodified.

21 baseline → 25 passing.

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach
Brian Krabach (bkrabach) merged commit 4490bf5 into main Aug 11, 2026
4 checks passed
Salil Das (sadlilas) pushed a commit that referenced this pull request Aug 18, 2026
…olution empty, not silent skip (#305)

When hooks-session-naming resolves a model_role and gets a clean empty result
(zero candidates), the code previously returned None and skipped session naming
entirely. The feature then silently never ran, making the system harder to
reason about and harder to test. A deterministic config gap should fail loud,
not silent.

This change falls back to the session's normal priority provider when role
resolution returns cleanly empty, logs a warning naming both the role that
failed to resolve and the provider actually substituted, then fires naming
once per session (subsequent calls drop to DEBUG).

This deliberately narrows PR #300 (fix(hooks): session-naming abort on
role-resolution failure, not silent downgrade) and does not undo it:

  - Resolver raises an exception → still aborts. Unchanged. An exception is
    an unknown, possibly transient failure; falling back could route onto the
    expensive model on every retry — the cost leak #300 guarded against.
  - Resolver returns cleanly empty → now falls back loudly. This is a
    deterministic config gap. Aborting means the feature never runs; falling
    back means it at least tries and admits what it did.

Related: This silent skip confounded a platform A/B test. The Linux run showed
zero errors not because the defect was absent, but because naming had silently
skipped and never exercised the provider code path. The Windows run had a fast
candidate, ran naming, and surfaced a real provider-lifecycle defect (PR #93).

Out of scope: If resolved is non-empty but no entry in providers matches the
resolved name, the loop falls through to the generic fallback with no warning
at all — a related silent gap left for follow-up.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-authored-by: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants