Skip to content

Implement streaming response optimization for non-Next.js publisher proxy #563

@aram356

Description

@aram356

Context

The publisher proxy currently buffers the entire response body in memory before sending any bytes to the client. For a 222KB HTML page, peak memory is ~4x the response size and no bytes reach the client until all processing completes.

Spec

See streaming response design spec (PR #562).

Implementation Steps

Step 1: Make the pipeline chunk-emitting

  • Rewrite HtmlRewriterAdapter to use lol_html incremental write() instead of accumulating full document
  • Convert process_gzip_to_gzip to use process_through_compression (chunk-based)
  • Convert decompress_and_process (used by *_to_none paths) to chunk-based processing
  • Fix process_through_compression to call encoder.finish() explicitly (prerequisite for gzip move)
  • Update existing tests for incremental output behavior

Step 2: Stream response to client

  • Refactor process_response_streaming to accept W: Write instead of Vec<u8>
  • Migrate main.rs from #[fastly::main] to undecorated main() with Request::from_client()
  • Implement streaming gate: html_post_processors().is_empty() for HTML; always stream non-HTML
  • Reorder synthetic ID/cookie and finalize_response() headers before stream_to_client()
  • Binary pass-through via io::copy(&mut body, &mut streaming_body)
  • Error handling: pre-stream errors → proper error response; mid-stream → log and abort

Validation

  • cargo test --workspace passes
  • Manual verification via fastly compute serve
  • Compare response bodies before/after for byte-identical output
  • Measure TTFB/TTLB and memory on staging

Acceptance Criteria

  • Streaming activates when Next.js is disabled and backend returns 2xx
  • Peak memory per request reduced from ~4x to constant (chunk buffer + parser state)
  • Client receives first body bytes after first processed chunk, not after full buffering
  • No regressions on static, auction, or discovery endpoints
  • Buffered fallback works correctly when post-processors are registered

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions