Skip to content

Commit ea11b3d

Browse files
committed
classifier 0.6.2 and kit 0.8.2: count a runner's own authentication retry
Routing section 9.3 treats an API key's authentication refusal as refused for any use only once it "persists after the runner's own retry". Kit 0.8.1's routed checks required exactly one provider call per selection for a 401, so a runner retrying once on the same account failed two cases on the count alone. Classifier CONTRACT section 6.8 now pins the bound: a runner's own retry of an authentication refusal is at most one more request on the same account and selected model, is not a reselection, and never moves the batch to another model. The kit accepts one or two calls per selection for the refused item, keeps answered items at one call and unsent items at zero, and requires a replay to add no provider call. The initial-failure case keeps 0.8.1's rule that its optional reselection may pick a new model. Self-tests accept one retry and reject two by check name.
1 parent 243a400 commit ea11b3d

10 files changed

Lines changed: 99 additions & 16 deletions

File tree

‎Cargo.lock‎

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎crates/cortexkit-role-classifier-conformance/CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
# Changelog
22

3+
## 0.8.2
4+
5+
- Pin classifier 0.6.2 and allow its runner-owned single authentication
6+
retry in the routed initial and partial account-failure checks, without
7+
allowing extra selections, model changes or provider calls on replay.
8+
- Keep answered items at one call and unsent items at zero. Add self-tests
9+
that accept one authentication retry and fail a runner that retries twice.
10+
311
## 0.8.1
412

513
- Pin classifier 0.6.1, whose catalog check accepts `decision` rows. The

‎crates/cortexkit-role-classifier-conformance/Cargo.toml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
# production build never compiles the suite.
88
[package]
99
name = "cortexkit-role-classifier-conformance"
10-
version = "0.8.1"
10+
version = "0.8.2"
1111
edition.workspace = true
1212
license.workspace = true
1313
repository.workspace = true
@@ -17,7 +17,7 @@ publish = true
1717

1818
[dependencies]
1919
async-trait = "0.1"
20-
cortexkit-role-classifier = { version = "=0.6.1", path = "../cortexkit-role-classifier" }
20+
cortexkit-role-classifier = { version = "=0.6.2", path = "../cortexkit-role-classifier" }
2121
futures-util = { version = "0.3", default-features = false }
2222
serde = { version = "1", features = ["derive"] }
2323
serde_jcs = "0.2"

‎crates/cortexkit-role-classifier-conformance/src/checks/routed.rs‎

Lines changed: 23 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -181,12 +181,12 @@ impl<S: ClassifierSubject> Suite<'_, S> {
181181
}
182182
let log = self.routing_log().await?;
183183
if !(1..=2).contains(&log.selections)
184-
|| log.provider_requests.len() != log.selections
185-
|| self.calls(&key).await != log.selections
184+
|| !(log.selections..=2 * log.selections).contains(&log.provider_requests.len())
185+
|| self.calls(&key).await != log.provider_requests.len()
186186
|| self.calls(&unsent_key).await != 0
187187
{
188188
return Err(
189-
"initial account failure exceeded one reselection or sent a later item".into(),
189+
"initial account failure exceeded one reselection or one authentication retry per selection, or sent a later item".into(),
190190
);
191191
}
192192
} else if name == "routed_partial_failure_keeps_model" {
@@ -212,20 +212,33 @@ impl<S: ClassifierSubject> Suite<'_, S> {
212212
return Err("partial account failure did not fail the unanswered items".into());
213213
}
214214
let log = self.routing_log().await?;
215+
let answered_calls = self.calls(&key).await;
216+
let refused_calls = self.calls(&failed_key).await;
217+
let unsent_calls = self.calls(&unsent_key).await;
215218
if log.selections != 1
216-
|| log.provider_requests != vec![first.clone(), first.clone()]
217-
|| self.calls(&unsent_key).await != 0
219+
|| answered_calls != 1
220+
|| !(1..=2).contains(&refused_calls)
221+
|| log.provider_requests.len() != answered_calls + refused_calls
222+
|| log
223+
.provider_requests
224+
.iter()
225+
.any(|decision| decision != first)
226+
|| unsent_calls != 0
218227
{
219-
return Err("partial account failure selected again or mixed models".into());
228+
return Err(
229+
"partial account failure selected again, mixed decisions or exceeded one authentication retry".into(),
230+
);
220231
}
221232
self.script_answer(&failed_key, noul_body("0.25", None))
222233
.await;
223234
let replay = Self::routed_reply(self.send(&request.to_string()).await?, first)?;
235+
let replay_log = self.routing_log().await?;
224236
if replay != reply
225-
|| self.routing_log().await?.selections != 1
226-
|| self.calls(&key).await != 1
227-
|| self.calls(&failed_key).await != 1
228-
|| self.calls(&unsent_key).await != 0
237+
|| replay_log.selections != log.selections
238+
|| replay_log.provider_requests != log.provider_requests
239+
|| self.calls(&key).await != answered_calls
240+
|| self.calls(&failed_key).await != refused_calls
241+
|| self.calls(&unsent_key).await != unsent_calls
229242
{
230243
return Err(
231244
"partial account failure was retried instead of replayed from storage".into(),

‎crates/cortexkit-role-classifier-conformance/tests/fake/mod.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,7 @@ struct RawItems {
270270

271271
pub struct FakeModule {
272272
routed_models: bool,
273+
auth_refusal_retries: usize,
273274
routing: Mutex<routed::RoutingState>,
274275
defects: Defects,
275276
priced: bool,
@@ -849,6 +850,11 @@ impl FakeSubject {
849850
subject
850851
}
851852

853+
pub fn with_auth_refusal_retries(mut self, retries: usize) -> Self {
854+
Arc::get_mut(&mut self.module).unwrap().auth_refusal_retries = retries;
855+
self
856+
}
857+
852858
fn build(
853859
defects: Defects,
854860
priced: bool,
@@ -859,6 +865,7 @@ impl FakeSubject {
859865
Self {
860866
module: Arc::new(FakeModule {
861867
routed_models: false,
868+
auth_refusal_retries: 0,
862869
routing: Mutex::new(routed::RoutingState::default()),
863870
defects,
864871
priced,

‎crates/cortexkit-role-classifier-conformance/tests/fake/routed.rs‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,18 @@ impl FakeModule {
9999
let answers = serde_json::from_str(answer.answers.get()).unwrap();
100100
items.push(ItemResult::answered(index as u64, answers));
101101
}
102-
Scripted::Status { .. } => {
102+
Scripted::Status { status, .. } => {
103+
if status == 401 {
104+
for _ in 0..self.auth_refusal_retries {
105+
self.routing
106+
.lock()
107+
.unwrap()
108+
.observations
109+
.provider_requests
110+
.push(used.clone());
111+
self.stand_in.call(&item.state).await;
112+
}
113+
}
103114
if index == 0 && self.defects.initial_account_failure_reselects_twice {
104115
for _ in 0..2 {
105116
let next = self.routing.lock().unwrap().select();

‎crates/cortexkit-role-classifier-conformance/tests/suite.rs‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,40 @@ async fn routed_initial_account_failure_cannot_select_twice() {
195195
.await;
196196
}
197197

198+
#[tokio::test]
199+
async fn routed_two_auth_retries_are_detected() {
200+
let subject = FakeSubject::routed(Defects::default()).with_auth_refusal_retries(2);
201+
let report = run(&subject).await;
202+
expect_breaks(
203+
&report,
204+
&[
205+
"routed_partial_failure_keeps_model",
206+
"routed_initial_account_failure_bounded",
207+
],
208+
"partial account failure",
209+
);
210+
}
211+
212+
#[tokio::test]
213+
async fn routed_single_auth_retry_passes_both_failure_cases() {
214+
let subject = FakeSubject::routed(Defects::default()).with_auth_refusal_retries(1);
215+
let report = run(&subject).await;
216+
assert_eq!(
217+
report.verdict,
218+
SuiteVerdict::PassedExceptNotApplicable {
219+
not_applicable: LEGACY_AUTH_CHECKS.to_vec()
220+
},
221+
"{}",
222+
report.render()
223+
);
224+
for check in [
225+
"routed_initial_account_failure_bounded",
226+
"routed_partial_failure_keeps_model",
227+
] {
228+
assert_eq!(report.outcome(check), Some(&CheckOutcome::Passed));
229+
}
230+
}
231+
198232
#[tokio::test]
199233
async fn faithful_routed_provider_passes_applicable_checks() {
200234
let report = run(&FakeSubject::routed(Defects::default())).await;

‎crates/cortexkit-role-classifier/CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
11
# Changelog
22

3+
## 0.6.2
4+
5+
- Clarify in §6.8 that a runner may retry an authentication refusal once on
6+
the same account and selected model under routing §9.3. This is not a
7+
reselection and never moves the batch to another model. Contract text
8+
only; no wire or Rust API changes.
9+
310
## 0.6.1
411

512
- A catalog row whose `kind` is `decision` now serves this role. That is

‎crates/cortexkit-role-classifier/CONTRACT.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -304,6 +304,9 @@ numbers, and say what is this role's own.
304304
- [pinned] A `routed_models` provider reports usage per decision and account
305305
failures to routing under the routing contract's `route.report` (§8) and
306306
`route.account_observe` (§9.2); their reporting rules apply unchanged.
307+
- [pinned] A runner's own retry of an authentication refusal (routing §9.3)
308+
is at most one more request on the same account and selected model, is not
309+
a reselection, and never moves the batch to another model.
307310

308311
### 6.9 Spend guard
309312

‎crates/cortexkit-role-classifier/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
# runner's production build never pulls in the suite.
1010
[package]
1111
name = "cortexkit-role-classifier"
12-
version = "0.6.1"
12+
version = "0.6.2"
1313
edition.workspace = true
1414
license.workspace = true
1515
repository.workspace = true

0 commit comments

Comments
 (0)