feat(pingora-core): expose PROXY v2 extension-TLV callback#22
Merged
Conversation
Adds an async filter (feature-gated) to adjust upstream modules prior to those modules (currently just compression) running.
This prevents headers like 100-continue from ending the stream and causing hangs while the downstream is waiting.
…pgrade When bootstrap_as_a_service is enabled, listen_fds() was called to snapshot the fd table before BootstrapService had run, always returning None. Services would then bind fresh sockets instead of inheriting the old process's fds, breaking graceful upgrades. Fix this by eagerly allocating the ListenFds table in Bootstrap::new() so it is non-optional and already distributed to all services before bootstrap runs. When load_fds() later receives the inherited fds from the old process, it populates the same shared table in place, making them visible to all services without any re-distribution.
This update introduces the abort_on_close feature to control behavior when a client closes the connection after the request body. When enabled (default), it results in a ConnectionClosed error, allowing the proxy to abort immediately. When disabled, the proxy can continue processing the upstream response. Includes-commit: a6420f8 Replicated-from: cloudflare#836
Add fields such that callers can distinguish a successful subrequest from one that died silently or was cut short. The handle lets callers await post-response cleanup (cache writes, logging) before issuing the next subrequest.
…istener port collisions test_conn_timeout / test_conn_timeout_with_offload: Replace 192.0.2.1 (TEST-~~~) with a bound-but-not-listening local socket via the new timeout_socket() helper in utils::for_testing. Because listen() is never called, the kernel silently drops SYN packets, guaranteeing a real ConnectTimedout on Linux. The total_connection_timeout tests still use 192.0.2.1 (SEMI_BLACKHOLE) since they test error classification and accept ConnectNoRoute as an alternative. test_tls_psk (s2n): PskTlsServer::start() spawned a background thread with no readiness check. Use an mpsc channel to signal after TcpListener::bind so tests only proceed once the port is ready. Also make the accept loop resilient to handshake failures (continue instead of panic) so a stale probe cannot take down the server. test_1xx_caching: mock_1xx_server used fixed ports (6151/6152) and sleep(100ms) for readiness. Refactored to spawn_mock_1xx_server which binds to port 0 (OS-assigned) and signals readiness via a oneshot channel after bind. Eliminates AddrInUse from TIME_WAIT and sleep races. test_listen_tcp / test_listen_tcp_ipv6_only: Hardcoded ports 7100-7102 collided across parallel CI test jobs. Switch to port 0 with the new ListenerEndpoint::local_addr() / Listener::local_addr() methods to discover the actual bound port.
ListenFds only guards an in-memory fd table and a blocking send_to_sock call, neither of which benefit from an async mutex. Switch to parking_lot::Mutex and move the fd-send path in main_loop onto the blocking thread pool via spawn_blocking. Because the parking_lot lock cannot be held across bind().await in ListenerEndpointBuilder::listen(), introduce a global per-address async lock map (flurry::HashMap<String, Arc<tokio::sync::Mutex<()>>>) that serializes the check-bind-insert sequence for each address. This prevents two concurrent callers from racing to bind the same address while the ListenFds lock is released.
The MSRV (1.84.0) job fails because cargo test compiles dev-dependencies. A transitive dev-dependency chain (pingora-proxy -> tokio-tungstenite -> tungstenite -> sha1 -> cpufeatures v0.3.0) pulls in a crate that uses edition 2024, which Cargo 1.84.0 cannot parse. Run cargo check --workspace for all toolchains and skip cargo test on the MSRV.
…ng graceful upgrades
Add BodyWriter task API (send_body_task, write_current_body_task, send_finish_task, write_current_finish_task) and HeaderWriter for cancel-safe writes that can be used in tokio::select! loops.
Using the proxy task API allows polling for the upstream rx task at the same time, so that upstream cache writes can continue even while serving downstream. proxy_h2 and h2 downstream (as well as custom) is a todo.
This was previously counting the response header bytes as well, which is incorrect.
Add LruUnit::peek_lru(), Lru::peek_lru(shard), and Manager::peek_lru(shard) to peek at the least-recently-used item in a shard without evicting it. Returns None for empty shards or out-of-bounds shard indices. This enables callers to report the eviction frontier — the age of the item that would be evicted next — for cache observability metrics.
Update bench_lru to test at production-level data sizes (~100K and ~500K items/shard). The original benchmark only tested 100 items across 10 shards (10 per shard), which made promote_top_n appear 42% faster. At larger scales, promote() is actually 20-25% faster because the read-lock scan rarely finds hot items near the head. Add heavy-hitter benchmarks (10 and 100 items at 10,000x weight) to test whether extremely concentrated access patterns benefit from promote_top_n. Result: promote() still ties or wins even with heavy hitters, because with few hot items spread across 32 shards, most shards have 0-1 hot items and the scan is wasted on cold accesses. Each benchmark variant uses a fresh LRU and a thread barrier to avoid state contamination and staggered starts. The 16M-item config is gated behind BENCH_LARGE=1 to avoid OOM on CI. Add a performance warning to promote_top_n() docs recommending promote() for large-scale workloads.
Also temp ignore the active RUSTSECs until the internal dependency bumps are synced.
Dictionary-compressed responses should vary on Available-Dictionary (RFC 9842) so caches don't serve them to mismatched clients. This adds the header in the compression module.
This is analogous to the downstream modules but can apply prior to upstream compression.
As opposed to panicking on an error while spawning a new stream, which may happen in rare situations if a server returns GOAWAY immediately upon creating the connection.
…stream is H2 When such a request reaches an H2 upstream, the existing version check (req.version != HTTP_2) may not fire if a malformed client sent hop-by-hop headers over H2. Add an is_custom() check so H1-specific headers are always stripped before forwarding to H2 when the downstream is a custom session.
This can happen when proxy tasks are enabled for downstream writes; an upstream miss handler error may end up disabling cache just as the downstream write finishes. In this and the non-proxy task case, the hit handler is dropped and no finish call should be made to begin with.
Bump dev-deps to pull in rustls-webpki 0.103.12.
The receiver drops when the coordinator exits the pipe loop, breaking the channel before the writer finishes its cache-write lifecycle. Return it in the state for callers to drain alongside the task handle.
…FC 9111 compliance)
Includes-commit: 875e4d9 Replicated-from: cloudflare#858 Signed-off-by: Shane Utt <shaneutt@linux.com>
The test connects to 240.0.0.1 (reserved) while bound to localhost and asserts the error is ConnectError or ConnectTimedout. On macOS and some CI runners the kernel returns ENETUNREACH immediately, which maps to ConnectNoRoute. Accept that as a valid outcome. This is the same class of fix applied to test_conn_timeout and test_tls_connect_timeout_supersedes_total in 542129f.
The old loop used `tokio::select!` with a `poll_closed` path that bailed as soon as the shutdown signal fired. RFC 9113 §6.8 says we have to process streams below the final last_stream_id. We weren't doing that. Now we call `graceful_shutdown` on the connection, but streams that were already in the buffer or have a lower stream number get surfaced and dispatched normally. The loop exits once the codec flushes the closing GOAWAY. This also pulls the accept loop out of `apps/mod.rs` so that it's more easily testable and usable from a test environment.
This is a trivially simple way to drive toward uniform weights between LRU shards if they are unbalanced.
This option is then passed to daemonize as the child process immediately runs chdir.
## Problem `test_upload_connection_die` fails reliably in CI on both arm64 and x86. The test sends a 15MB upload to an nginx origin that immediately responds with 200, then kills the connection after 1s. Under CI load, the 15MB upload takes longer than 1s. When nginx sends the TCP RST, it discards the buffered 200 response (per TCP protocol semantics). The proxy sees an upstream error and resets the client connection, causing the test to fail with `ConnectionReset`. This is not a test bug — the proxy does not reliably forward early responses while still writing the request body upstream. The `select!` loop in `proxy_handle_upstream` is blocked on `send_body_to1` and cannot read the response concurrently. ## Fix Mark the test as `#[ignore]` with a detailed comment explaining the root cause.
Implement the same proxy task API functionality for subrequest server sessions as HTTP/1. Also fix the regular subrequest header write path so upgrade state is only marked after the 101 task is sent.
Creates ListenerConfig to hold this new config and allow for future extensibility.
`maybe_consume_proxy_header` already parses PROXY v2 headers in
`UninitializedStream::handshake` and threads the recovered source
address into the SocketDigest, but it silently drops every parsed
extension TLV. Consumer apps that ride application-defined metadata
through the same header (HAProxy v2 spec § 2.2 reserves type IDs
0xE0..=0xEF for that) had no way to receive them.
Add a global callback registration parallel to the existing
`set_client_hello_callback`:
pub type ProxyV2TlvCallback =
Option<fn(&[ExtensionTlv], SocketAddr)>;
pub fn set_proxy_v2_tlv_callback(callback: ProxyV2TlvCallback);
`maybe_consume_proxy_header` invokes the callback with the parsed
`extensions` slice and the recovered source `SocketAddr` whenever
the PROXY v2 header carried any TLVs. No-op when the callback isn't
registered or the TLV list is empty, so existing deployments are
unaffected.
Re-exports `proxy_protocol::version2::ExtensionTlv` through
`crate::protocols::proxy_protocol::ExtensionTlv` so callbacks can
pattern-match on `ExtensionTlv::Custom { type_id, value }` without
depending on the underlying proxy-protocol crate directly.
Depends on the `Custom` variant added in
gen0sec/proxy-protocol#12 — pingora-core's proxy-protocol dep is
temporarily pinned to that branch; flip back to `main` once the PR
lands.
Use case: synapse-proxy's TLS-passthrough edge will encode per-flow
JA4 fingerprints as a 0xE0 Custom TLV in the v2 header it already
emits; the Tier-2 proxy receives them via this callback and
populates its fingerprint cache without an out-of-band store.
gen0sec/synapse#352.
CI's `cargo fmt --all -- --check` flagged two multi-line items in listeners/mod.rs that rustfmt's default policy collapses to a single line (the `ProxyV2TlvCallback` type alias and the `call_proxy_v2_tlv_callback` function signature). Both fit within the 100-column limit.
gen0sec/proxy-protocol#12 (the `ExtensionTlv::Custom` variant this callback depends on) shipped as v0.5.3. Drop the temporary branch reference and pin to the tagged release.
Author
CI status update (
|
`proxy-protocol >= 0.5.3` declares `edition = "2024"` in its manifest, which Cargo only stabilized in Rust 1.85.0. Building pingora-core against the v0.5.3 proxy-protocol release with rustc 1.84.0 fails at manifest parse: feature `edition2024` is required The package requires the Cargo feature called `edition2024`, but that feature is not stabilized in this version of Cargo (1.84.0 (66221abde 2024-11-19)). Bumping the MSRV pin in the build matrix to 1.85.0 picks up the stabilized edition without forcing proxy-protocol to revert to edition 2021.
`test_connect_proxying_allowed_h1` and `test_connect_proxying_disallowed_h1` in `pingora-proxy/tests/test_basic.rs` were failing on the fork's main branch (no relation to this PR's listener changes) with: Fail to proxy: Downstream InvalidHTTPHeader context: invalid uri pingora.org:443 The CONNECT request-line carries an authority-form URI (`pingora.org:443`, RFC 9110 § 9.3.6) rather than origin-form (`/path?query`). `RequestHeader::set_raw_path` was building the URI via `Uri::builder().path_and_query(...)`, which only accepts origin-form and rejects authority-form with `PathDoesNotStartWithSlash`. That short-circuited the request before the CONNECT-method handler in `pingora-proxy/src/lib.rs:260-274` could return 405 (for the disallowed test) or before the proxy could tunnel the bytes (for the allowed test). Fix: - `set_raw_path` tries the permissive `path_and_query` builder first (preserves the looser byte handling existing callers depend on for paths like `\`), then falls back to `Uri::try_from(...)` which auto-detects authority / absolute / asterisk forms. - `raw_path()` no longer unwraps `path_and_query()`. Authority-form URIs have no `path_and_query`; we return the authority bytes (e.g. `pingora.org:443`). Asterisk-form falls through to an empty slice rather than panicking. Verified with `cargo test -p pingora-proxy --test test_basic test_connect_proxying` — 3/3 pass after this change.
Follow-up to the CONNECT authority-form fix. CI bailed at the test
step before reaching later stages, so these were only surfaced once
the CONNECT tests passed:
1. `test_single_header` / `test_multiple_header` use `b"\\"` as a
request path. `http >= 1.4` rejects paths that don't start with
`/` (PathDoesNotStartWithSlash) where `http <= 1.3` accepted them,
so `RequestHeader::build("GET", b"\\")` started erroring. Make
`set_raw_path` preserve such bytes in `raw_path_fallback` (the
mechanism the non-UTF8 branch already uses) with a `/` sentinel
on `base.uri`, instead of erroring. raw_path() / the H1 wire
serializer read the fallback first, so the original bytes still
round-trip.
2. `set_proxy_v2_tlv_callback` doctest moved the non-Copy `real_addr`
inside a loop (E0382). Borrow it instead.
3. `test_single_header_no_case` used `for_each(|_| unreachable!())`,
which clippy's `never_loop` flags. Replaced with
`assert!(iter.next().is_none())`.
Verified on the CI toolchain (1.91.1): pingora-http --lib (8 pass),
pingora-core --doc (3 pass), CONNECT tests (3 pass), and
`cargo +1.91.1 clippy --all-targets --all -- --deny=warnings` clean.
…allback # Conflicts: # pingora-core/src/listeners/mod.rs # pingora-core/src/tls/mod.rs
pigri
added a commit
to gen0sec/synapse
that referenced
this pull request
Jun 1, 2026
gen0sec/pingora#22 merged into main, bringing the PROXY v2 TLV callback, the reqwest 0.12 / rustls 0.23 upstream sync, and pinning proxy-protocol to the v0.5.3 release. Bump synapse's proxy-protocol [patch] entries v0.5.2 → v0.5.3 so they resolve to the exact same git source the merged pingora-core pulls — otherwise the two tags are distinct packages and Cargo rejects the duplicate `links = "proxy-protocol"`. Builds clean against merged pingora main (the upstream sync's hyper 1.0 / TLS-module restructure / set_buffer signature change don't affect synapse's pingora API usage).
pigri
added a commit
to gen0sec/synapse
that referenced
this pull request
Jun 1, 2026
gen0sec/pingora#22 merged into main, bringing the PROXY v2 TLV callback, the reqwest 0.12 / rustls 0.23 upstream sync, and pinning proxy-protocol to the v0.5.3 release. Bump synapse's proxy-protocol [patch] entries v0.5.2 → v0.5.3 so they resolve to the exact same git source the merged pingora-core pulls — otherwise the two tags are distinct packages and Cargo rejects the duplicate `links = "proxy-protocol"`. Builds clean against merged pingora main (the upstream sync's hyper 1.0 / TLS-module restructure / set_buffer signature change don't affect synapse's pingora API usage).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
maybe_consume_proxy_headerinUninitializedStream::handshakealready parses PROXY v2 headers and threads the recovered sourceSocketAddrinto the SocketDigest, but it silently drops every parsed extension TLV. Consumer apps that need to ride application-defined metadata through the same header (HAProxy v2 spec § 2.2 reserves type IDs0xE0..=0xEFfor downstream consumer use) currently have no path to receive them.Adds a global callback registration parallel to the existing
set_client_hello_callback:maybe_consume_proxy_headerinvokes the callback with the parsedextensionsslice and the recovered sourceSocketAddrwhenever the PROXY v2 header carried any TLVs. No-op when the callback isn't registered or the TLV list is empty — existing deployments unaffected.Re-exports
ExtensionTlvthroughcrate::protocols::proxy_protocol::ExtensionTlvso callbacks can pattern-match onExtensionTlv::Custom { type_id, value }without taking a direct dep on the underlyingproxy_protocolcrate.Dependency
Pulls the
ExtensionTlv::Customvariant from gen0sec/proxy-protocol#12.pingora-core/Cargo.tomlis temporarily pinned to that branch; flip back tomain(or a tagged version) once it lands.Use case
synapse-proxy's TLS-passthrough edge proxy will encode per-flow JA4 fingerprints into a
0xE0Custom TLV on the v2 header it already emits. The Tier-2 proxy receives them via this callback and populates its fingerprint cache, so the WAF / access log / rate-limiter can see JA4 even though the TLS bytes never reach the Tier-2 proxy in cleartext.Tracking: gen0sec/synapse#352.
Test plan
cargo build -p pingora-coreclean against the patched proxy-protocol branch