Retry transient Claude model discovery failures

[andy-xu-db]

Why

AI Gateway model discovery can return HTTP 429 when the workspace-level models-list rate limit is exceeded. Both ucode setup and the runtime Claude Code model-discovery proxy must retry so discovery remains available under a short burst.

Related gateway change: https://github.com/databricks-eng/universe/pull/2514436

Changes

• Add opt-in retries to the JSON GET helper while preserving one attempt for existing callers by default.
• Allow two setup-time retries for ucode Anthropic model discovery.
• Retry HTTP 429 and network failures, honor numeric Retry-After, and use bounded exponential backoff with jitter.
• Retry runtime GET /v1/models HTTP 429 responses once inside the Claude discovery proxy before relaying a failure to Claude Code. The single retry keeps the request within Claude Code's roughly three-second discovery deadline.
• Drain retryable responses before reusing the upstream connection; inference requests remain unchanged.

Testing

• Targeted unit tests: 260 passed.
• Full test suite: 2107 passed, 37 skipped.
• Repository-wide Ruff checks: passed.
• Direct UC permission checks succeeded for Sonnet 4.6, Opus 5, and Haiku 4.5.

Actual Claude Code client retry probe

Ran Claude Code 2.1.251 against a deterministic upstream through the ucode proxy. The upstream returned 429 once and then 200; inference returned OK and Claude Code exited 0.

[ucode-relay] {"attempt":1,"elapsed_ms":2,"event":"model_discovery_upstream_headers","status":429}
[ucode-relay] {"attempt":2,"delay_ms":1033,"event":"model_discovery_retry_scheduled"}
[ucode-relay] {"attempt":2,"elapsed_ms":1036,"event":"model_discovery_upstream_headers","status":200}
[ucode-relay] {"elapsed_ms":1037,"event":"model_discovery_response_complete","status":200}
claude_version=2.1.251 (Claude Code)
exit_code=0
stdout='OK'
models_requests=2
messages_requests=1

LiteSwap E2E (gateway rate limit set to 200 RPS)

Tested against the gateway change through testenv://liteswap/andy-aigw-models-rl in staging-aws-us-east-1-0.

• A 450-request concurrent burst to the models-list endpoint produced 255 HTTP 200 responses and 195 HTTP 429 responses; rejected requests returned REQUEST_LIMIT_EXCEEDED.
• Ran actual ucode Claude Code 2.1.251 through the LiteSwap: runtime GET /v1/models received HTTP 429, ucode retried with bounded backoff, the retry returned HTTP 200 with 12 models, and Sonnet 4.6 inference returned HTTP 200 with output OK and process exit code 0.

[lilly-luo]

would be good to see client-side debug logs or video or smth of the retry