GitLab

The context-error rewrite (Anthropic-native phrasing) is necessary but not
sufficient: vLLM reports the overflow "input length" as a lower bound equal to
(context + 1 - max_tokens), not the true prompt size. The Claude Agent SDK's
self-heal retries with max_tokens = context - input - 1000, so every retry that
shrinks max_tokens is met by vLLM raising the reported input by the same amount
— the sum stays pinned one token over the limit and the retry never converges,
exhausting all 11 SDK attempts and hard-failing.

Fix: custom_callbacks.py now over-reports the input by a relative factor
(_CONTEXT_OVERCOUNT_FACTOR = 1.05, +5%) in the rewritten error, so each retry
shrinks max_tokens faster than vLLM's lower bound climbs and the SDK lands on a
fitting max_tokens within 2-3 attempts. Relative (not a fixed delta) so it
scales with proximity to the ceiling and self-corrects across retries. Scoped to
the x-agent-context-error: anthropic header, so only opted-in agent callers are
affected.

Verified end-to-end against the live proxy: a ~236k-token request that
previously exhausted all 11 attempts now self-heals on the first retry
(is_error=false). Near-ceiling prompts (input within ~4k of the limit) correctly
bail and must compact — that zone is owned by the agent-side auto_compact_window.

Deployed to /opt/litellm on nocode (backup: custom_callbacks.py.bak-pre-overcount-20260703).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>