litellm: purposeful input over-count so SDK context self-heal converges
Problem
Yesterday's context-error rewrite (translating vLLM's OpenAI-style overflow 400 into the Anthropic-native input length and \max_tokens` exceed context limit: + > phrasing) was **necessary but not sufficient**. Agent sessions onqwen3.6-35b` still hard-failed on context overflow.
Root cause (reproduced live): vLLM reports the overflow "input length" as a lower bound = context + 1 − max_tokens, not the true prompt size. The Claude Agent SDK's self-heal retries with max_tokens = context − input − 1000. So every retry that shrinks max_tokens is met by vLLM raising the reported input by the same amount — the sum stays pinned one token over the limit and the retry never converges, exhausting all 11 SDK attempts:
attempt 1/11: 230145 + 32000 > 262144
attempt 2/11: 231146 + 30999 > 262144
...
attempt 11/11: 240155 + 21990 > 262144 → hard fail (every row = 262145)Fix
custom_callbacks.py now deliberately over-reports the input by a relative factor (_CONTEXT_OVERCOUNT_FACTOR = 1.05, +5%) in the rewritten error. Each retry then shrinks max_tokens faster than vLLM's lower bound climbs, so the SDK lands on a fitting max_tokens within 2–3 attempts. Relative (not a fixed delta) so it scales with proximity to the ceiling and self-corrects across retries. Scoped to the x-agent-context-error: anthropic header — plain clients see the untouched error.
Validation (end-to-end against the live proxy)
| Prompt | True input | Room | Before | After |
|---|---|---|---|---|
| mid-range | ~236k tok | ~26k | exhausts 11 attempts, hard-fail | self-heals on first retry, is_error=false |
| near-ceiling | ~246k tok | ~16k | hard-fail | bails → must compact (correct) |
Near-ceiling prompts (true input within ~4k of the limit) correctly bail — there's no room for even the minimum output — and must compact. That zone is owned by the agent-side auto_compact_window provider setting, not this hook.
Deploy state
Already applied to /opt/litellm/custom_callbacks.py on nocode and LiteLLM restarted (backup: custom_callbacks.py.bak-pre-overcount-20260703). This MR brings the source of truth in line. README updated with the rationale.