fix: Django setup timing, auto app-ready, and urllib3 botocore support

[sohankshirsagar]

Summary

Fixes to improve Django onboarding reliability, reduce manual setup, and resolve urllib3 instrumentation failures with botocore/boto3.

1. Django middleware injection timing fix

When TuskDrift.initialize() is called at the top of a Django manage.py (as our setup instructions recommend), Django settings are not yet configured — DJANGO_SETTINGS_MODULE hasn't been set and django.setup() hasn't run. The middleware injection silently failed, meaning DriftMiddleware was never added to the middleware stack. Without it, no SERVER spans (kind 2) were created — only CLIENT spans (kind 3) from outbound instrumentations (requests, psycopg2, redis). Since tusk drift list filters for SERVER spans to identify traces, it reported "No traces found" even though the app was running and handling requests normally.

The root cause is a timing issue:

1. SDK initializes first (top of file, before any Django setup)
2. DjangoInstrumentation.patch() checks settings.configured → False → gives up
3. The registry marks the module as __drift_patched__, so the patch is never retried
4. Django starts up and serves requests without DriftMiddleware in the stack

The fix defers middleware injection to django.setup() when settings aren't available at patch time.

2. Auto-mark app as ready on first inbound request

Previously, apps had to explicitly call TuskDrift.get_instance().mark_app_as_ready() (e.g. in Django's AppConfig.ready()). If this was missed, all spans were recorded with isPreAppStart: true and the SDK never transitioned to normal recording mode.

Now, all three framework handlers (Django middleware, FastAPI, WSGI/Flask) automatically call mark_app_as_ready() on the first inbound HTTP request if the app hasn't been marked ready yet. This acts as a safety net — explicit mark_app_as_ready() calls still work and take precedence (unless they are called after first inbound request).

3. Skip non-HTTP schemes in urllib instrumentation

The urllib instrumentation was intercepting all urlopen() calls, including file:// URLs. Libraries like xmlschema use urlopen("file:///path/to/schema.xsd") to read local files. In REPLAY mode, the SDK tried to find a mock for these local file reads and crashed with RuntimeError: No mock found likely bc these spans were filtered out due to size.

The fix checks the URL scheme early and passes through anything that isn't http:// or https://. This is safe because file://, data://, ftp://, etc. are local operations.

4. Fix urllib3 instrumentation for botocore/boto3 (preload_content=False)

botocore uses preload_content=False for all HTTP requests via urllib3, which means the response body isn't read during construction — callers use response.read() later. This caused three cascading failures:

• Bytes header handling: botocore/urllib3 can produce headers with bytes keys/values instead of str. This caused TypeError: a bytes-like object is required, not 'str' in _get_decoded_type_from_content_type() and would break JSON serialization of headers. Fixed by adding _normalize_headers() and decoding bytes in content-type helpers.

• Response body not captured during recording: The old code skipped body capture entirely for preload_content=False responses to "avoid breaking the application." This meant replay had no body to return, causing 500 errors. Fixed by reading the raw bytes from response._fp and replacing _fp with a BytesIO so the caller's subsequent read() still works. A 1 MB size guard prevents buffering very large streaming downloads.

• Mock response broken during replay: _create_mock_response() used preload_content=True, which exhausted the BytesIO during construction. urllib3's read() does not check _body — only the .data property does. So botocore's response.read() got b"", CRC32 checksum was 0, and ChecksumError was raised. Fixed by using preload_content=False for mock responses, leaving the BytesIO unread in _fp for the caller.

Also added exc_info=True to the catch-all error logger in _finalize_span for better debugging.

E2e tests added for the preload_content=False pattern: read() (botocore pattern), read() + CRC32 validation (DynamoDB pattern), and stream() (chunked reading). Previously these were excluded from the replay test suite.

[cubic-dev-ai]

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="drift/instrumentation/django/instrumentation.py">

<violation number="1" location="drift/instrumentation/django/instrumentation.py:127">
P1: Restoring `django.setup` before calling it makes the deferred middleware hook racey on threaded startup.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[tusk-dev]

Generated 25 tests - 25 passed

 

Tip

New to Tusk Unit Tests? Learn more here.

Test Summary

• DriftMiddleware.__call__ - 8 ✓
• patched_open - 9 ✓
• handle_wsgi_request - 8 ✓

Results

Tusk's tests are all passing and cover three critical fixes in this PR. The Django middleware auto-ready tests validate that DriftMiddleware.__call__ properly marks the app as ready on first inbound request in RECORD and REPLAY modes, with idempotency checks ensuring it only fires once. The urllib scheme filtering tests confirm that patched_open correctly bypasses instrumentation for non-HTTP schemes (file://, data://, ftp://) while still instrumenting http:// and https:// URLs, both as strings and Request objects—this prevents crashes when libraries like xmlschema read local files. The WSGI handler auto-ready tests mirror the Django middleware coverage, ensuring handle_wsgi_request auto-marks the app on first request in RECORD and REPLAY modes with proper ordering before downstream logic. Together, these tests validate the two main behavioral changes: automatic app readiness marking across all framework handlers (eliminating manual setup), and safe passthrough of non-HTTP schemes in urllib instrumentation (preventing RuntimeError: No mock found crashes in REPLAY mode).

View check history

Commit	Status	Output	Created (UTC)
b694610	Generated 33 tests - 33 passed	Tests	Mar 18, 2026 10:55PM
38d378f	Generated 25 tests - 25 passed	Tests	Mar 19, 2026 5:43PM

Was Tusk helpful? Give feedback by reacting with 👍 or 👎

[cubic-dev-ai]

1 issue found across 6 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="drift/instrumentation/urllib3/instrumentation.py">

<violation number="1" location="drift/instrumentation/urllib3/instrumentation.py:933">
P1: Missing `_normalize_headers()` in `_try_get_mock` — replay mode will crash with `TypeError` when botocore passes bytes headers. The same bytes-header fix applied to `_finalize_span` needs to be applied here.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="drift/instrumentation/urllib3/instrumentation.py">

<violation number="1" location="drift/instrumentation/urllib3/instrumentation.py:908">
P1: Only remove `content-encoding` after successful decompression. Right now failed/unsupported decompression still strips the header, which can corrupt replayed responses.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="drift/instrumentation/urllib3/instrumentation.py">

<violation number="1" location="drift/instrumentation/urllib3/instrumentation.py:1009">
P1: `Content-Encoding` lookup is case-sensitive, so compressed responses may not be decompressed before recording/replay.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}