-
Notifications
You must be signed in to change notification settings - Fork 1
Add support for deflate, brotli, and zstd compression formats #30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The debug toolbar middleware was failing to inject the toolbar HTML into gzip-compressed responses. When Litestar's compression middleware runs before the debug toolbar, the response body is gzip bytes, which cannot be decoded as UTF-8. The middleware silently returned the original body without injection. This fix: - Detects gzip content-encoding header - Decompresses the response body before injection - Injects the toolbar HTML into the decompressed HTML - Returns uncompressed response (strips content-encoding header) This ensures the toolbar is visible when compression is enabled, which is the default in many production-like configurations. Fixes toolbar not appearing on admin pages when compression is enabled. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com>
Co-authored-by: JacobCoffee <45884264+JacobCoffee@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR extends the debug toolbar middleware to support multiple compression formats beyond gzip, addressing feedback from PR #24. The implementation adds support for deflate (zlib), Brotli (br), and Zstandard (zstd) compression formats, with optional dependencies for Brotli and Zstandard.
Key Changes:
- Added decompression support for deflate, br (Brotli), and zstd formats with graceful fallback for missing optional libraries
- Refactored decompression logic into a dedicated
_decompress_body()method for better code organization - Added comprehensive test coverage for all compression formats including edge cases and error handling
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
src/debug_toolbar/litestar/middleware.py |
Implements multi-format decompression support with _decompress_body() method, adds optional imports for brotli and zstandard, updates _inject_toolbar() signature to return tuple with encoding info |
tests/integration/test_litestar_middleware.py |
Adds TestMultipleCompressionFormats class with 6 tests covering deflate, brotli, zstd, multiple encodings, invalid data handling, and missing library scenarios |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| class TestMultipleCompressionFormats: | ||
| """Test toolbar injection with various compression formats.""" | ||
|
|
||
| def test_toolbar_injected_with_deflate_compression(self) -> None: | ||
| """Test that toolbar is correctly injected when response is deflate-compressed.""" | ||
| import zlib | ||
|
|
||
| @get("/deflate", media_type=MediaType.HTML) | ||
| async def deflate_handler() -> Response: | ||
| """Return deflate-compressed HTML.""" | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| compressed = zlib.compress(html.encode("utf-8")) | ||
| return Response( | ||
| content=compressed, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "deflate"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[deflate_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/deflate") | ||
| assert response.status_code == 200 | ||
| assert b"debug-toolbar" in response.content | ||
| assert b"</body>" in response.content | ||
|
|
||
| def test_toolbar_injected_with_brotli_compression(self) -> None: | ||
| """Test that toolbar is correctly injected when response is brotli-compressed.""" | ||
| try: | ||
| import brotli | ||
| except ImportError: | ||
| pytest.skip("brotli library not available") | ||
|
|
||
| @get("/brotli", media_type=MediaType.HTML) | ||
| async def brotli_handler() -> Response: | ||
| """Return brotli-compressed HTML.""" | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| compressed = brotli.compress(html.encode("utf-8")) | ||
| return Response( | ||
| content=compressed, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "br"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[brotli_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/brotli") | ||
| assert response.status_code == 200 | ||
| assert b"debug-toolbar" in response.content | ||
| assert b"</body>" in response.content | ||
|
|
||
| def test_toolbar_injected_with_zstd_compression(self) -> None: | ||
| """Test that toolbar is correctly injected when response is zstd-compressed.""" | ||
| try: | ||
| import zstandard | ||
| except ImportError: | ||
| pytest.skip("zstandard library not available") | ||
|
|
||
| @get("/zstd", media_type=MediaType.HTML) | ||
| async def zstd_handler() -> Response: | ||
| """Return zstd-compressed HTML.""" | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| cctx = zstandard.ZstdCompressor() | ||
| compressed = cctx.compress(html.encode("utf-8")) | ||
| return Response( | ||
| content=compressed, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "zstd"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[zstd_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/zstd") | ||
| assert response.status_code == 200 | ||
| assert b"debug-toolbar" in response.content | ||
| assert b"</body>" in response.content | ||
|
|
||
| def test_toolbar_with_multiple_encodings(self) -> None: | ||
| """Test handling of multiple comma-separated encodings.""" | ||
|
|
||
| @get("/multi-encoding", media_type=MediaType.HTML) | ||
| async def multi_encoding_handler() -> Response: | ||
| """Return HTML with multiple encodings applied.""" | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| # Apply gzip first | ||
| compressed = gzip.compress(html.encode("utf-8")) | ||
| return Response( | ||
| content=compressed, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "gzip, identity"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[multi_encoding_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/multi-encoding") | ||
| assert response.status_code == 200 | ||
| assert b"debug-toolbar" in response.content | ||
| assert b"</body>" in response.content | ||
|
|
||
| def test_invalid_deflate_data_with_deflate_header(self) -> None: | ||
| """Test handling of invalid deflate data with content-encoding: deflate header.""" | ||
|
|
||
| @get("/invalid-deflate", media_type=MediaType.HTML) | ||
| async def invalid_deflate_handler() -> Response: | ||
| """Return invalid deflate data with deflate content-encoding header.""" | ||
| invalid_deflate = b"This is not deflated data but pretends to be" | ||
| return Response( | ||
| content=invalid_deflate, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "deflate"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[invalid_deflate_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/invalid-deflate") | ||
| assert response.status_code == 200 | ||
| # Should return original content since it couldn't be decompressed | ||
| assert b"This is not deflated data but pretends to be" in response.content | ||
|
|
||
| def test_brotli_without_library(self) -> None: | ||
| """Test that responses with brotli encoding are handled gracefully without the library.""" | ||
| try: | ||
| import brotli # noqa: F401 | ||
|
|
||
| pytest.skip("brotli library is available, can't test missing library case") | ||
| except ImportError: | ||
| pass | ||
|
|
||
| @get("/brotli-missing", media_type=MediaType.HTML) | ||
| async def brotli_missing_handler() -> Response: | ||
| """Return content with br encoding when library is not available.""" | ||
| # Since brotli isn't available, we can't actually compress it | ||
| # This simulates a response claiming to be brotli-compressed | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| return Response( | ||
| content=html.encode("utf-8"), | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "br"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[brotli_missing_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/brotli-missing") | ||
| assert response.status_code == 200 | ||
| # Should handle gracefully and attempt to inject toolbar | ||
| # (will work since the data isn't actually compressed) | ||
| assert b"Test" in response.content |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no test coverage for handling unknown or unsupported compression encodings (e.g., "compress", "identity", or future formats). The decompression loop silently skips unknown encodings, which could lead to unintended behavior.
Consider adding a test case that verifies the middleware handles unknown encodings gracefully, such as a response with content-encoding: compress or content-encoding: unknown-format.
| dctx = zstandard.ZstdDecompressor() | ||
| body = dctx.decompress(body) |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A new ZstdDecompressor instance is created for every zstd decompression operation. For frequently accessed compressed responses, this could have a performance impact.
Consider instantiating the decompressor once at the class or module level (when zstandard is available) and reusing it, or verify if the overhead is negligible. The brotli library uses brotli.decompress() as a module-level function, which is more efficient.
| else: | ||
| # Zstandard not available, can't decompress | ||
| logger.debug("Zstandard compression detected but zstandard library not installed") | ||
| break |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The _decompress_body method doesn't handle the "identity" encoding, which indicates no compression. When processing encodings like "gzip, identity", the loop will process "identity" (after reversing the list), but since there's no case for it, the code will skip it and move to the next iteration without updating the decompressed flag.
According to HTTP specs, "identity" means the content is already uncompressed, so it should be skipped or explicitly handled. Consider adding an elif encoding == "identity": continue case to make the intent clear and avoid confusion.
| break | |
| break | |
| elif encoding == "identity": | |
| # "identity" means no encoding; body is already uncompressed | |
| continue |
| for encoding in reversed(encodings): | ||
| if encoding == "gzip": | ||
| try: | ||
| body = gzip.decompress(body) | ||
| decompressed = True | ||
| except (gzip.BadGzipFile, OSError): | ||
| # Not valid gzip, try next encoding or decode as-is | ||
| break | ||
| elif encoding == "deflate": | ||
| try: | ||
| body = zlib.decompress(body) | ||
| decompressed = True | ||
| except zlib.error: | ||
| # Not valid deflate, try next encoding or decode as-is | ||
| break | ||
| elif encoding == "br": | ||
| if brotli is not None: | ||
| try: | ||
| body = brotli.decompress(body) | ||
| decompressed = True | ||
| except brotli.error: # type: ignore[attr-defined] | ||
| # Not valid brotli, try next encoding or decode as-is | ||
| break | ||
| else: | ||
| # Brotli not available, can't decompress | ||
| logger.debug("Brotli compression detected but brotli library not installed") | ||
| break | ||
| elif encoding == "zstd": | ||
| if zstandard is not None: | ||
| try: | ||
| dctx = zstandard.ZstdDecompressor() | ||
| body = dctx.decompress(body) | ||
| decompressed = True | ||
| except zstandard.ZstdError: | ||
| # Not valid zstd, try next encoding or decode as-is | ||
| break |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The break statements in the error handling for decompression failures will stop processing remaining encodings in the list. This could be problematic if multiple encodings were applied and one fails.
For example, if we have "gzip, deflate" and the gzip decompression fails (invalid data), the code will break without attempting deflate decompression. Consider whether continue would be more appropriate to skip failed encodings and try the next one, or if breaking is the intended behavior. If breaking is intentional, this should be documented in the docstring.
| If compression was decompressed, returns uncompressed body with empty encoding. | ||
| """ | ||
| # Handle compressed responses | ||
| encodings = [e.strip() for e in content_encoding.lower().split(",")] if content_encoding else [] |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The encodings list includes empty strings when the content-encoding header value ends with a comma (e.g., "gzip,") or contains double commas (e.g., "gzip,,deflate"). The strip() operation on line 562 removes whitespace but not empty strings resulting from split.
Consider filtering out empty strings from the encodings list to avoid unnecessary loop iterations: encodings = [e.strip() for e in content_encoding.lower().split(",") if e.strip()]
| encodings = [e.strip() for e in content_encoding.lower().split(",")] if content_encoding else [] | |
| encodings = [e.strip() for e in content_encoding.lower().split(",") if e.strip()] if content_encoding else [] |
| def test_toolbar_with_multiple_encodings(self) -> None: | ||
| """Test handling of multiple comma-separated encodings.""" | ||
|
|
||
| @get("/multi-encoding", media_type=MediaType.HTML) | ||
| async def multi_encoding_handler() -> Response: | ||
| """Return HTML with multiple encodings applied.""" | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| # Apply gzip first | ||
| compressed = gzip.compress(html.encode("utf-8")) | ||
| return Response( | ||
| content=compressed, | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "gzip, identity"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[multi_encoding_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/multi-encoding") | ||
| assert response.status_code == 200 | ||
| assert b"debug-toolbar" in response.content | ||
| assert b"</body>" in response.content |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test applies only gzip compression but sets the content-encoding header to "gzip, identity". This doesn't accurately test multiple encodings being applied in sequence. According to HTTP spec, when multiple encodings are listed (e.g., "gzip, deflate"), each encoding should be applied sequentially to the data, but this test only applies gzip.
Consider either:
- Renaming the test to clarify it's testing parsing of comma-separated values rather than actual multiple encoding layers, or
- Actually applying multiple compression layers to properly test the reverse decompression logic
| elif encoding == "br": | ||
| if brotli is not None: | ||
| try: | ||
| body = brotli.decompress(body) | ||
| decompressed = True | ||
| except brotli.error: # type: ignore[attr-defined] | ||
| # Not valid brotli, try next encoding or decode as-is | ||
| break | ||
| else: | ||
| # Brotli not available, can't decompress | ||
| logger.debug("Brotli compression detected but brotli library not installed") | ||
| break | ||
| elif encoding == "zstd": | ||
| if zstandard is not None: | ||
| try: | ||
| dctx = zstandard.ZstdDecompressor() | ||
| body = dctx.decompress(body) | ||
| decompressed = True | ||
| except zstandard.ZstdError: | ||
| # Not valid zstd, try next encoding or decode as-is | ||
| break | ||
| else: | ||
| # Zstandard not available, can't decompress | ||
| logger.debug("Zstandard compression detected but zstandard library not installed") | ||
| break |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a compression library is not available (brotli or zstandard), the code logs a debug message and breaks from the decompression loop. However, after breaking, the decompressed flag remains False even though the method returns the original body unchanged.
This could be confusing because the function signature suggests decompressed=False means "decompression failed", but in this case it means "decompression wasn't attempted". The calling code in _inject_toolbar will then try to decode the body as UTF-8 and potentially inject the toolbar into still-compressed data. Consider logging a warning instead of debug, or returning the original encoding so the response can be passed through unchanged.
| def test_brotli_without_library(self) -> None: | ||
| """Test that responses with brotli encoding are handled gracefully without the library.""" | ||
| try: | ||
| import brotli # noqa: F401 | ||
|
|
||
| pytest.skip("brotli library is available, can't test missing library case") | ||
| except ImportError: | ||
| pass | ||
|
|
||
| @get("/brotli-missing", media_type=MediaType.HTML) | ||
| async def brotli_missing_handler() -> Response: | ||
| """Return content with br encoding when library is not available.""" | ||
| # Since brotli isn't available, we can't actually compress it | ||
| # This simulates a response claiming to be brotli-compressed | ||
| html = "<html><body><h1>Test</h1></body></html>" | ||
| return Response( | ||
| content=html.encode("utf-8"), | ||
| status_code=HTTP_200_OK, | ||
| media_type=MediaType.HTML, | ||
| headers={"content-encoding": "br"}, | ||
| ) | ||
|
|
||
| config = LitestarDebugToolbarConfig(enabled=True) | ||
| app = Litestar( | ||
| route_handlers=[brotli_missing_handler], | ||
| plugins=[DebugToolbarPlugin(config)], | ||
| debug=True, | ||
| ) | ||
| with TestClient(app) as client: | ||
| response = client.get("/brotli-missing") | ||
| assert response.status_code == 200 | ||
| # Should handle gracefully and attempt to inject toolbar | ||
| # (will work since the data isn't actually compressed) | ||
| assert b"Test" in response.content |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a test for handling missing brotli library (test_brotli_without_library), but no equivalent test for missing zstandard library. Since zstandard is treated as an optional dependency in the middleware (similar to brotli), there should be a corresponding test to verify graceful handling when zstandard is not available.
Consider adding a test_zstd_without_library test that mirrors the brotli test structure.
| import brotli # noqa: F401 | ||
|
|
||
| pytest.skip("brotli library is available, can't test missing library case") | ||
| except ImportError: |
Copilot
AI
Dec 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
'except' clause does nothing but pass and there is no explanatory comment.
| except ImportError: | |
| except ImportError: | |
| # brotli is not installed; continue to run the test for the missing-library scenario. |
The middleware only handled gzip-compressed responses. Other common HTTP compression formats (deflate, br, zstd) would fail toolbar injection.
Changes
Middleware (
src/debug_toolbar/litestar/middleware.py):zlib(stdlib)brotlilibraryzstandardlibrary_decompress_body()helper to handle all formatsTests (
tests/integration/test_litestar_middleware.py):TestMultipleCompressionFormatsclass with 6 test casesDocumentation (
docs/configuration.md):Behavior
Decompressed responses return uncompressed with empty
Content-Encodingheader.💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.