Skip to content

Add streaming HTTP response support to gremlin-driver#3419

Open
Cole-Greer wants to merge 2 commits intomasterfrom
java-streaming
Open

Add streaming HTTP response support to gremlin-driver#3419
Cole-Greer wants to merge 2 commits intomasterfrom
java-streaming

Conversation

@Cole-Greer
Copy link
Copy Markdown
Contributor

Summary

Adds streaming HTTP response deserialization to gremlin-driver for GraphBinary. Instead of buffering the entire HTTP response body before processing, the driver now deserializes results incrementally as data arrives from the server.

Design

Four new classes in gremlin-driver:

  • ByteBufQueueInputStream — Bridges the Netty event loop and a reader thread via a BlockingQueue of ByteBufs. The event loop offers HTTP content chunks; the reader thread consumes them through standard InputStream reads.
  • InputStreamBuffer — Read-only Buffer implementation backed by an InputStream. Allows existing TypeSerializer implementations to work unchanged over a streaming response body.
  • GraphBinaryStreamResponseReader — Reads the GraphBinary v4 wire format one item at a time, pushing each result to the ResultSet as it is deserialized.
  • HttpStreamingResponseHandler — Netty handler that replaces HttpObjectAggregator + HttpGremlinResponseDecoder in the pipeline for GraphBinary responses.

Non-GraphBinary serializers (GraphSON, custom MessageSerializer implementations) fall back to the previous aggregating pipeline.

A new cached thread pool (gremlin-driver-stream-reader) is added to Cluster for the blocking reader threads, bounded by connection pool size.

Cole-Greer added 2 commits May 4, 2026 15:17
Adds incremental GraphBinary response deserialization to the Java driver.
HTTP response chunks are fed to a BlockingQueue-backed InputStream, and a
dedicated reader thread deserializes items one at a time using the existing
TypeSerializer infrastructure via an InputStream-backed Buffer adapter.

Results are pushed to the ResultSet as they are deserialized, enabling
consumers to process results before the full response is received.

Non-GraphBinary serializers (GraphSON, custom MessageSerializer) fall back
to the previous aggregating pipeline (HttpObjectAggregator +
HttpGremlinResponseDecoder).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant