Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 28, 2025

The middleware only handled gzip-compressed responses. Other common HTTP compression formats (deflate, br, zstd) would fail toolbar injection.

Changes

Middleware (src/debug_toolbar/litestar/middleware.py):

  • Added deflate support via zlib (stdlib)
  • Added brotli support via optional brotli library
  • Added zstd support via optional zstandard library
  • Extracted _decompress_body() helper to handle all formats
  • Process comma-separated encodings in reverse order per HTTP spec
  • Graceful degradation when optional libraries unavailable

Tests (tests/integration/test_litestar_middleware.py):

  • New TestMultipleCompressionFormats class with 6 test cases
  • Coverage for each format, invalid data, missing libraries, multiple encodings

Documentation (docs/configuration.md):

  • Compression support section with format table
  • Installation instructions for optional dependencies

Behavior

# Before: Only gzip worked
Content-Encoding: gzip  # ✓ Toolbar injected
Content-Encoding: deflate  # ✗ Failed silently

# After: All formats supported
Content-Encoding: gzip  # ✓
Content-Encoding: deflate  # ✓
Content-Encoding: br  # ✓ (if brotli installed)
Content-Encoding: zstd  # ✓ (if zstandard installed)
Content-Encoding: gzip, identity  # ✓ Handles multiple encodings

Decompressed responses return uncompressed with empty Content-Encoding header.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

JacobCoffee and others added 11 commits December 27, 2025 21:20
The debug toolbar middleware was failing to inject the toolbar HTML into
gzip-compressed responses. When Litestar's compression middleware runs
before the debug toolbar, the response body is gzip bytes, which cannot
be decoded as UTF-8. The middleware silently returned the original body
without injection.

This fix:
- Detects gzip content-encoding header
- Decompresses the response body before injection
- Injects the toolbar HTML into the decompressed HTML
- Returns uncompressed response (strips content-encoding header)

This ensures the toolbar is visible when compression is enabled, which
is the default in many production-like configurations.

Fixes toolbar not appearing on admin pages when compression is enabled.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Base automatically changed from fix/gzip-compression-injection to main December 28, 2025 04:11
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com>
@JacobCoffee JacobCoffee marked this pull request as ready for review December 28, 2025 04:16
Copilot AI review requested due to automatic review settings December 28, 2025 04:16
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com>
Copilot AI changed the title [WIP] WIP address feedback on gzip-compressed responses Add support for deflate, brotli, and zstd compression formats Dec 28, 2025
Copilot AI requested a review from JacobCoffee December 28, 2025 04:18
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the debug toolbar middleware to support multiple compression formats beyond gzip, addressing feedback from PR #24. The implementation adds support for deflate (zlib), Brotli (br), and Zstandard (zstd) compression formats, with optional dependencies for Brotli and Zstandard.

Key Changes:

  • Added decompression support for deflate, br (Brotli), and zstd formats with graceful fallback for missing optional libraries
  • Refactored decompression logic into a dedicated _decompress_body() method for better code organization
  • Added comprehensive test coverage for all compression formats including edge cases and error handling

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
src/debug_toolbar/litestar/middleware.py Implements multi-format decompression support with _decompress_body() method, adds optional imports for brotli and zstandard, updates _inject_toolbar() signature to return tuple with encoding info
tests/integration/test_litestar_middleware.py Adds TestMultipleCompressionFormats class with 6 tests covering deflate, brotli, zstd, multiple encodings, invalid data handling, and missing library scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +461 to +642
class TestMultipleCompressionFormats:
"""Test toolbar injection with various compression formats."""

def test_toolbar_injected_with_deflate_compression(self) -> None:
"""Test that toolbar is correctly injected when response is deflate-compressed."""
import zlib

@get("/deflate", media_type=MediaType.HTML)
async def deflate_handler() -> Response:
"""Return deflate-compressed HTML."""
html = "<html><body><h1>Test</h1></body></html>"
compressed = zlib.compress(html.encode("utf-8"))
return Response(
content=compressed,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "deflate"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[deflate_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/deflate")
assert response.status_code == 200
assert b"debug-toolbar" in response.content
assert b"</body>" in response.content

def test_toolbar_injected_with_brotli_compression(self) -> None:
"""Test that toolbar is correctly injected when response is brotli-compressed."""
try:
import brotli
except ImportError:
pytest.skip("brotli library not available")

@get("/brotli", media_type=MediaType.HTML)
async def brotli_handler() -> Response:
"""Return brotli-compressed HTML."""
html = "<html><body><h1>Test</h1></body></html>"
compressed = brotli.compress(html.encode("utf-8"))
return Response(
content=compressed,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "br"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[brotli_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/brotli")
assert response.status_code == 200
assert b"debug-toolbar" in response.content
assert b"</body>" in response.content

def test_toolbar_injected_with_zstd_compression(self) -> None:
"""Test that toolbar is correctly injected when response is zstd-compressed."""
try:
import zstandard
except ImportError:
pytest.skip("zstandard library not available")

@get("/zstd", media_type=MediaType.HTML)
async def zstd_handler() -> Response:
"""Return zstd-compressed HTML."""
html = "<html><body><h1>Test</h1></body></html>"
cctx = zstandard.ZstdCompressor()
compressed = cctx.compress(html.encode("utf-8"))
return Response(
content=compressed,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "zstd"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[zstd_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/zstd")
assert response.status_code == 200
assert b"debug-toolbar" in response.content
assert b"</body>" in response.content

def test_toolbar_with_multiple_encodings(self) -> None:
"""Test handling of multiple comma-separated encodings."""

@get("/multi-encoding", media_type=MediaType.HTML)
async def multi_encoding_handler() -> Response:
"""Return HTML with multiple encodings applied."""
html = "<html><body><h1>Test</h1></body></html>"
# Apply gzip first
compressed = gzip.compress(html.encode("utf-8"))
return Response(
content=compressed,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "gzip, identity"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[multi_encoding_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/multi-encoding")
assert response.status_code == 200
assert b"debug-toolbar" in response.content
assert b"</body>" in response.content

def test_invalid_deflate_data_with_deflate_header(self) -> None:
"""Test handling of invalid deflate data with content-encoding: deflate header."""

@get("/invalid-deflate", media_type=MediaType.HTML)
async def invalid_deflate_handler() -> Response:
"""Return invalid deflate data with deflate content-encoding header."""
invalid_deflate = b"This is not deflated data but pretends to be"
return Response(
content=invalid_deflate,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "deflate"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[invalid_deflate_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/invalid-deflate")
assert response.status_code == 200
# Should return original content since it couldn't be decompressed
assert b"This is not deflated data but pretends to be" in response.content

def test_brotli_without_library(self) -> None:
"""Test that responses with brotli encoding are handled gracefully without the library."""
try:
import brotli # noqa: F401

pytest.skip("brotli library is available, can't test missing library case")
except ImportError:
pass

@get("/brotli-missing", media_type=MediaType.HTML)
async def brotli_missing_handler() -> Response:
"""Return content with br encoding when library is not available."""
# Since brotli isn't available, we can't actually compress it
# This simulates a response claiming to be brotli-compressed
html = "<html><body><h1>Test</h1></body></html>"
return Response(
content=html.encode("utf-8"),
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "br"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[brotli_missing_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/brotli-missing")
assert response.status_code == 200
# Should handle gracefully and attempt to inject toolbar
# (will work since the data isn't actually compressed)
assert b"Test" in response.content
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no test coverage for handling unknown or unsupported compression encodings (e.g., "compress", "identity", or future formats). The decompression loop silently skips unknown encodings, which could lead to unintended behavior.

Consider adding a test case that verifies the middleware handles unknown encodings gracefully, such as a response with content-encoding: compress or content-encoding: unknown-format.

Copilot uses AI. Check for mistakes.
Comment on lines +537 to +538
dctx = zstandard.ZstdDecompressor()
body = dctx.decompress(body)
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new ZstdDecompressor instance is created for every zstd decompression operation. For frequently accessed compressed responses, this could have a performance impact.

Consider instantiating the decompressor once at the class or module level (when zstandard is available) and reusing it, or verify if the overhead is negligible. The brotli library uses brotli.decompress() as a module-level function, which is more efficient.

Copilot uses AI. Check for mistakes.
else:
# Zstandard not available, can't decompress
logger.debug("Zstandard compression detected but zstandard library not installed")
break
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _decompress_body method doesn't handle the "identity" encoding, which indicates no compression. When processing encodings like "gzip, identity", the loop will process "identity" (after reversing the list), but since there's no case for it, the code will skip it and move to the next iteration without updating the decompressed flag.

According to HTTP specs, "identity" means the content is already uncompressed, so it should be skipped or explicitly handled. Consider adding an elif encoding == "identity": continue case to make the intent clear and avoid confusion.

Suggested change
break
break
elif encoding == "identity":
# "identity" means no encoding; body is already uncompressed
continue

Copilot uses AI. Check for mistakes.
Comment on lines +507 to +542
for encoding in reversed(encodings):
if encoding == "gzip":
try:
body = gzip.decompress(body)
decompressed = True
except (gzip.BadGzipFile, OSError):
# Not valid gzip, try next encoding or decode as-is
break
elif encoding == "deflate":
try:
body = zlib.decompress(body)
decompressed = True
except zlib.error:
# Not valid deflate, try next encoding or decode as-is
break
elif encoding == "br":
if brotli is not None:
try:
body = brotli.decompress(body)
decompressed = True
except brotli.error: # type: ignore[attr-defined]
# Not valid brotli, try next encoding or decode as-is
break
else:
# Brotli not available, can't decompress
logger.debug("Brotli compression detected but brotli library not installed")
break
elif encoding == "zstd":
if zstandard is not None:
try:
dctx = zstandard.ZstdDecompressor()
body = dctx.decompress(body)
decompressed = True
except zstandard.ZstdError:
# Not valid zstd, try next encoding or decode as-is
break
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The break statements in the error handling for decompression failures will stop processing remaining encodings in the list. This could be problematic if multiple encodings were applied and one fails.

For example, if we have "gzip, deflate" and the gzip decompression fails (invalid data), the code will break without attempting deflate decompression. Consider whether continue would be more appropriate to skip failed encodings and try the next one, or if breaking is the intended behavior. If breaking is intentional, this should be documented in the docstring.

Copilot uses AI. Check for mistakes.
If compression was decompressed, returns uncompressed body with empty encoding.
"""
# Handle compressed responses
encodings = [e.strip() for e in content_encoding.lower().split(",")] if content_encoding else []
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The encodings list includes empty strings when the content-encoding header value ends with a comma (e.g., "gzip,") or contains double commas (e.g., "gzip,,deflate"). The strip() operation on line 562 removes whitespace but not empty strings resulting from split.

Consider filtering out empty strings from the encodings list to avoid unnecessary loop iterations: encodings = [e.strip() for e in content_encoding.lower().split(",") if e.strip()]

Suggested change
encodings = [e.strip() for e in content_encoding.lower().split(",")] if content_encoding else []
encodings = [e.strip() for e in content_encoding.lower().split(",") if e.strip()] if content_encoding else []

Copilot uses AI. Check for mistakes.
Comment on lines +555 to +581
def test_toolbar_with_multiple_encodings(self) -> None:
"""Test handling of multiple comma-separated encodings."""

@get("/multi-encoding", media_type=MediaType.HTML)
async def multi_encoding_handler() -> Response:
"""Return HTML with multiple encodings applied."""
html = "<html><body><h1>Test</h1></body></html>"
# Apply gzip first
compressed = gzip.compress(html.encode("utf-8"))
return Response(
content=compressed,
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "gzip, identity"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[multi_encoding_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/multi-encoding")
assert response.status_code == 200
assert b"debug-toolbar" in response.content
assert b"</body>" in response.content
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test applies only gzip compression but sets the content-encoding header to "gzip, identity". This doesn't accurately test multiple encodings being applied in sequence. According to HTTP spec, when multiple encodings are listed (e.g., "gzip, deflate"), each encoding should be applied sequentially to the data, but this test only applies gzip.

Consider either:

  1. Renaming the test to clarify it's testing parsing of comma-separated values rather than actual multiple encoding layers, or
  2. Actually applying multiple compression layers to properly test the reverse decompression logic

Copilot uses AI. Check for mistakes.
Comment on lines +522 to +546
elif encoding == "br":
if brotli is not None:
try:
body = brotli.decompress(body)
decompressed = True
except brotli.error: # type: ignore[attr-defined]
# Not valid brotli, try next encoding or decode as-is
break
else:
# Brotli not available, can't decompress
logger.debug("Brotli compression detected but brotli library not installed")
break
elif encoding == "zstd":
if zstandard is not None:
try:
dctx = zstandard.ZstdDecompressor()
body = dctx.decompress(body)
decompressed = True
except zstandard.ZstdError:
# Not valid zstd, try next encoding or decode as-is
break
else:
# Zstandard not available, can't decompress
logger.debug("Zstandard compression detected but zstandard library not installed")
break
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a compression library is not available (brotli or zstandard), the code logs a debug message and breaks from the decompression loop. However, after breaking, the decompressed flag remains False even though the method returns the original body unchanged.

This could be confusing because the function signature suggests decompressed=False means "decompression failed", but in this case it means "decompression wasn't attempted". The calling code in _inject_toolbar will then try to decode the body as UTF-8 and potentially inject the toolbar into still-compressed data. Consider logging a warning instead of debug, or returning the original encoding so the response can be passed through unchanged.

Copilot uses AI. Check for mistakes.
Comment on lines +609 to +642
def test_brotli_without_library(self) -> None:
"""Test that responses with brotli encoding are handled gracefully without the library."""
try:
import brotli # noqa: F401

pytest.skip("brotli library is available, can't test missing library case")
except ImportError:
pass

@get("/brotli-missing", media_type=MediaType.HTML)
async def brotli_missing_handler() -> Response:
"""Return content with br encoding when library is not available."""
# Since brotli isn't available, we can't actually compress it
# This simulates a response claiming to be brotli-compressed
html = "<html><body><h1>Test</h1></body></html>"
return Response(
content=html.encode("utf-8"),
status_code=HTTP_200_OK,
media_type=MediaType.HTML,
headers={"content-encoding": "br"},
)

config = LitestarDebugToolbarConfig(enabled=True)
app = Litestar(
route_handlers=[brotli_missing_handler],
plugins=[DebugToolbarPlugin(config)],
debug=True,
)
with TestClient(app) as client:
response = client.get("/brotli-missing")
assert response.status_code == 200
# Should handle gracefully and attempt to inject toolbar
# (will work since the data isn't actually compressed)
assert b"Test" in response.content
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a test for handling missing brotli library (test_brotli_without_library), but no equivalent test for missing zstandard library. Since zstandard is treated as an optional dependency in the middleware (similar to brotli), there should be a corresponding test to verify graceful handling when zstandard is not available.

Consider adding a test_zstd_without_library test that mirrors the brotli test structure.

Copilot uses AI. Check for mistakes.
import brotli # noqa: F401

pytest.skip("brotli library is available, can't test missing library case")
except ImportError:
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
except ImportError:
except ImportError:
# brotli is not installed; continue to run the test for the missing-library scenario.

Copilot uses AI. Check for mistakes.
@JacobCoffee JacobCoffee closed this Jan 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants