fix: retry transient 'thinking blocks cannot be modified' 400s from Bedrock

[seanturner83]

Summary

Bedrock occasionally returns HTTP 400 claiming the assistant's thinking blocks have been modified, even when the payload is structurally valid. The identical payload replayed immediately succeeds. This is a transient server-side condition, not a client bug.

Observed on bedrock/us.anthropic.claude-sonnet-4-6 with adaptive thinking (reasoning_effort=high → {type: "adaptive"} on claude-4-6). Error text:

messages.N.content.M: `thinking` or `redacted_thinking` blocks in the latest
assistant message cannot be modified. These blocks must remain as they were
in the original response.

Fix

• Add _is_transient_thinking_error() that matches on status_code == 400 and the characteristic message ("thinking" + "cannot be modified"). Deliberately strict to avoid hijacking unrelated 400s.
• On detection, retry with exponential backoff: 5s, 10s, 20s, 40s, 80s, 160s (~5 min total budget) before falling through to generic retry.
• Self-contained detector — works independently of other 400 handlers.

Evidence

• Replayed 33 captured payloads from around an observed failure: all succeeded on replay, confirming transience.
• In a multi-repo variance study (21+ scans × 7 repos of Sonnet 4.6 on real production code), the 3-retry budget was exhausted in one case (~30s). A 6-retry budget succeeded on a subsequent attempt with the same repo, same model, same config.

Test plan

• Backoff schedule unit-check: 5s, 10s, 20s, 40s, 80s, 160s; total 315s
• Detector smoke test: correctly matches a BedrockException with the thinking-blocks message, rejects oversize-payload 400s and generic 400s
• No change to behaviour when the detector returns False (fall-through to existing retry path)

[greptile-apps]

Greptile Summary

This PR adds a targeted retry path for Bedrock's transient "thinking blocks cannot be modified" 400 errors, using a dedicated _is_transient_thinking_error detector and exponential back-off (5–160 s, up to 6 retries).

• Retry budget is not independent of max_retries: the continue in the transient handler advances the outer for attempt loop, so each transient retry consumes a generic-retry slot. With the default max_retries=5 the loop runs 6 total iterations, the 6th back-off (160 s) never fires, and the documented "fall through to generic retry" path is never reached. A separate inner loop is needed to keep the two retry budgets independent.

Confidence Score: 3/5

The fix improves resilience against the observed transient error but does not behave as documented with default configuration — the full 6-retry budget and generic-retry fallthrough are unreachable.

A P1 logic defect: the transient retry counter shares the outer attempt slots, so max_retries=5 caps effective transient retries at 5 (not 6) and prevents generic fallthrough. The change is otherwise self-contained and additive.

strix/llm/llm.py — generate method retry loop (lines 176–181)

Important Files Changed

Filename	Overview
strix/llm/llm.py	Adds transient-thinking-error detector and retry handler, but the retry loop shares the outer attempt counter, limiting effective retries to min(max_retries, 6)=5 with defaults and making the "fall through to generic retry" path unreachable.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: strix/llm/llm.py
Line: 176-181

Comment:
**Transient retries share the outer `attempt` budget**

The `continue` on line 181 advances the `for attempt in range(max_retries + 1)` counter, so each transient thinking retry consumes one generic-retry slot. With the default `max_retries=5` the loop runs for attempts 0–5 (6 iterations total). On attempt 5, `attempt >= max_retries` is `True`, so the `_raise_error` branch fires before the 6th sleep — the documented 160 s back-off never runs and the "fall through to generic retry" path is unreachable with default config.

The fix as described in the PR (independent 6-retry budget that falls through to generic retry) requires the transient counter to drive its own inner loop without consuming outer `attempt` slots, e.g.:

```python
if self._is_transient_thinking_error(e):
    if transient_thinking_retries < 6:
        transient_thinking_retries += 1
        await asyncio.sleep(min(240, 5 * (2 ** (transient_thinking_retries - 1))))
        continue  # retry WITHOUT advancing attempt
    # exhausted transient budget; fall through to generic retry below
```

But `continue` in a `for` loop always advances the iterator — a separate `while True` inner retry loop is needed to keep `attempt` from incrementing.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "fix: retry transient 'thinking blocks ca..." | Re-trigger Greptile}