discovery,transport: discv5 peering + replace lsquic with zquic v1.2.1#42
Merged
Conversation
Implements the full discovery layer design for zig-ethp2p, grounded in the ethp2p spec §Peering and §Transport design decisions. ENR layer (src/discovery/enr/): - enr.zig: RLP encode/decode, Enr struct with key-value lookup, EIP-778 300B limit - standard.zig: standard field decoders (secp256k1, ip, udp, tcp) - ethp2p.zig: eth-ec capability field (version, EC scheme bitmask, geo-hint, custody columns); EthEcField.supportsScheme() ties to layer/ec_scheme.zig discv5 layer (src/discovery/discv5/): - table.zig: 256-bucket Kademlia routing table (k=16), XOR distance, closest-N query, bucket LRS eviction - crypto.zig: AES-128-GCM encrypt/decrypt (std.crypto), HKDF-SHA256 session key derivation; secp256k1 ECDH stubbed with TODO (requires BoringSSL) - packet.zig: Ordinary / WHOAREYOU / Handshake wire type constants and masking-key derivation (SHA256) - session.zig: per-peer session state (idle/awaiting/established), nonce counter, SessionTable keyed by NodeId - protocol.zig: PING/PONG/FINDNODE/NODES/TALKREQ/TALKRES message types and protocol constants (max_nodes_per_response=4, request_timeout_ms=500) - node.zig: poll-driven drive loop (timer expiry, bucket refresh), bootstrap, closest lookup, capability-aware peer query Peering layer (src/discovery/peering/): - score.zig: RTT-first composite scoring with EMA smoothing, event-triggered updates, time-based half-life decay; latency tiers (inner<60ms, mid<120ms, outer) aligned with broadcast chunk dispatch ordering - duty.zig: DutyKind enum (proposer/attester/aggregator/sync/ptc/das); per-duty slot capacities (16/64/16/8/16/128 selfish + 50 altruistic); connection pool tier capacities (20/80/100); SelectionReq per duty type - table.zig: SelfishTable (evicts lowest-scoring when full), AltruisticTable, combined PeerTable; selectForDuty with RTT filter - pool.zig: connection pool tracking hot/warm/cold warmth state with 0-RTT session ticket storage - warmup.zig: 12s slot phases (block 0-2s, attest 2-4s, agg 4-8s, idle 8-12s); lookahead_slots=2 warmup scheduler; requests accepted only during idle phase All modules have tests. Wired into src/root.zig as pub const discovery. secp256k1 is the only unimplemented primitive (clearly stubbed with TODO).
Three functions that were previously stubbed are now implemented using
the BoringSSL EC_KEY / ECDH APIs that are already vendored via lsquic_zig:
- generateEphemeralKeypair: EC_KEY_new_by_curve_name(NID_secp256k1) +
EC_KEY_generate_key, extracts compressed pubkey and raw scalar.
- ecdhSharedSecret: BN_bin2bn + EC_POINT_oct2point + ECDH_compute_key,
returns the raw x-coordinate (32 bytes) as required by discv5 §6.
- nodeIdFromPubkey: decompresses to 65-byte uncompressed form then hashes
bytes [1..] with std.crypto.sha3.Keccak256 per discv5 §4.1.
build.zig: store the BoringSSL openssl_include LazyPath in QuicLinkBundle
and propagate it to the root zig_ethp2p module via wireZigEthP2pModule so
that @cImport("openssl/...") in crypto.zig resolves when -Denable-quic is
set. Without the flag ossl compiles as an empty struct and the three
functions return Secp256k1Error, which the tests skip via SkipZigTest.
New tests: keygen+ECDH roundtrip (verifies ECDH(a,B)==ECDH(b,A)) and
nodeId uniqueness, both gated on build_opts.enable_quic.
secp256k1 is a fundamental discv5 primitive — it is needed regardless of whether the QUIC transport shim is active. Previously it was gated on -Denable-quic which forced callers to pass a flag unrelated to discovery. build.zig: - Split QuicLinkBundle into BoringSslBundle (always built, contains ssl_lib, crypto_lib, openssl_include) + QuicLinkBundle (lsquic only, still behind -Denable-quic). - addBoringSsl() extracts BoringSSL from lsquic_zig unconditionally. - addLsquicQuicModule() takes BoringSslBundle and adds lsquic on top. - wireZigEthP2pModule() always adds openssl_include to the root module. - linkCryptoLibs() (new) links ssl+crypto into every compile step. - linkQuicLibs() now only links lsquic + zlib/pthread/m. crypto.zig: - Drop build_opts import and all if (build_opts.enable_quic) guards. - @cImport is unconditional — headers are always available. - generateEphemeralKeypair, ecdhSharedSecret, nodeIdFromPubkey are fully implemented with no runtime stubs. - Tests no longer have SkipZigTest guards; all four tests run in every plain `zig build test` invocation.
…uic flag lsquic and BoringSSL are fundamental — secp256k1 (discv5) and QUIC transport are both always needed. Keeping a conditional flag created unnecessary complexity and stub code. build.zig: - Single LsquicBundle replaces the two-struct BoringSslBundle/QuicLinkBundle split. addLsquicBundle() builds everything unconditionally. - wireModule() and linkLibs() have no optional parameters. - The -Denable-quic b.option() is removed entirely; so is the zig_opts / zig_ethp2p_options module (its only consumer was enable_quic). - Windows targets panic at build time (lsquic_zig is not Windows focused). eth_ec_quic.zig: - All if (comptime !build_opts.enable_quic) stubs removed. - build_opts import removed. - EthEcQuicListener.inner flattened to direct ep/port fields. - listen/dial/pollListener/logInit call through directly with no guards. - Last test block unconditionally pulls in eth_ec_quic_enabled.zig. Docs and CI: - README: QUIC transport section no longer marked optional; -Denable-quic removed from build table, build examples, and all anchors. - ci.yml / justfile: test-quic command drops -Denable-quic flag. - UPSTREAM.md, root.zig, eth_ec_quic_enabled.zig, ci_root_quic.zig, crypto.zig: comments updated to remove flag references. Result: plain `zig build test` now runs all 77 tests including the full QUIC handshake and stream-framing integration tests.
… std.crypto secp256k1 ECDSA and ECDH are now implemented using Zig 0.15's own std.crypto.ecc.Secp256k1 and std.crypto.sign.ecdsa, eliminating the BoringSSL C FFI from crypto.zig entirely. Changes by file: - crypto.zig: drop @cImport; use EcdsaK=Ecdsa(Secp256k1,Keccak256); streaming signer/verifier for id-nonce multi-part hash; pure Zig ECDH via point.mul + affineCoordinates().x - enr/enr.zig: fix rlpDecode to return full item (prefix + content); add rlpEncodeUint64, rlpEncodeList, EnrBuilder.sign, verifyV4 - discv5/packet.zig: full ORDINARY/WHOAREYOU/HANDSHAKE codec with AES-128-CTR masking; fix decode to unmask entire header in one CTR pass - discv5/protocol.zig: complete RLP encode/decode for all six message types (PING/PONG/FINDNODE/NODES/TALKREQ/TALKRES) - discv5/table.zig: add getEntry and refreshNode helpers - discv5/node.zig: UDP socket, non-blocking recv loop, iterative FINDNODE, queryByCapability, bucket refresh timer - peering/pool.zig: free session_ticket when promoteHot overwrites warm entry - peering/score.zig: fix signed-integer division and u8 saturating add - discovery/peer_manager.zig (new): bridges discv5 → warmup scheduler → eth_ec_quic.dial → PeerTable registration and score-based eviction - transport/eth_ec_quic_peer.zig: implement dispatchInboundUniStream with on_sess_stream / on_chunk_stream callbacks; cancel unknown streams - src/root.zig: add discovery to the test block (was missing) 141 tests pass, 0 leaks.
… std.crypto secp256k1 ECDSA and ECDH are now implemented using Zig 0.15's own std.crypto.ecc.Secp256k1 and std.crypto.sign.ecdsa, eliminating the BoringSSL C FFI from crypto.zig entirely. Changes by file: - crypto.zig: drop @cImport; use EcdsaK=Ecdsa(Secp256k1,Keccak256); streaming signer/verifier for id-nonce multi-part hash; pure Zig ECDH via point.mul + affineCoordinates().x - enr/enr.zig: fix rlpDecode to return full item (prefix + content); add rlpEncodeUint64, rlpEncodeList, EnrBuilder.sign, verifyV4 - discv5/packet.zig: full ORDINARY/WHOAREYOU/HANDSHAKE codec with AES-128-CTR masking; fix decode to unmask entire header in one CTR pass - discv5/protocol.zig: complete RLP encode/decode for all six message types (PING/PONG/FINDNODE/NODES/TALKREQ/TALKRES) - discv5/table.zig: add getEntry and refreshNode helpers - discv5/node.zig: UDP socket, non-blocking recv loop, iterative FINDNODE, queryByCapability, bucket refresh timer - peering/pool.zig: free session_ticket when promoteHot overwrites warm entry - peering/score.zig: fix signed-integer division and u8 saturating add - transport/eth_ec_quic_peer.zig: implement dispatchInboundUniStream with on_sess_stream / on_chunk_stream callbacks; cancel unknown streams - src/root.zig: add discovery to the test block (was missing) 141 tests pass, 0 leaks.
Discovery crypto (secp256k1, ECDSA, ECDH) now uses pure std.crypto, so wireModule no longer needs to add the openssl include path. BoringSSL headers are only required by lsquic_quic_shim.zig.
…ile errors Add an optional `on_peer_dialed` callback (with opaque context pointer) to `PeerManager` so that callers can react when a QUIC dial succeeds. The callback is fired from `dialPeer` before the peer is registered in the connection pool. Also fix three pre-existing compile errors that prevented the module from building: - `duty_mod.score_eviction_floor` did not exist; add a local constant (-100) - `discv5_node.table` is not `pub`; change to a direct import of `discv5/table.zig` - `std.time.nanoTimestamp()` returns `i128` but `promoteHot` expects `u64`; add `@intCast`
Add SharedUdpSocket (src/transport/shared_udp_socket.zig) that owns a single bound UDP socket and demultiplexes inbound datagrams between the discv5 discovery layer and the lsquic QUIC transport: - Try discv5 first via Node.injectDatagram (new); AES header decode reliably rejects non-discv5 packets, returning false so the caller can forward to QUIC. - Forward rejected packets to lsquic via feedPacket (new public wrapper around lsquic_engine_packet_in). - Run QUIC timer processing separately via processEngineOnly (new). Node changes (discv5/node.zig): - Add owns_socket bool to track fd ownership. - Add startFromFd(fd) — attach external fd without binding. - stop/deinit skip close when owns_socket = false. - poll skips recvLoop when using external fd. - Add pub injectDatagram(data, from) -> bool. lsquic_quic_shim.zig: - Add owns_sock bool to QuicEndpoint. - Add endpointInitFromFd — create engine on existing fd. - endpointDeinit respects owns_sock. - Add feedPacket and processEngineOnly. eth_ec_quic.zig + eth_ec_quic_enabled.zig: - Add listenOnFd, feedPacket, processEngineOnly wrappers. - Re-export QuicEndpoint so callers outside the package can name it.
Drop the lsquic C shim and all BoringSSL/zlib/pthread link steps. The QUIC stack is now zquic (https://github.com/ch4r10t33r/zquic), fetched as a Zig package from the v1.2.1 release tarball. New shim (zquic_quic_shim.zig) re-implements the same QuicEndpoint / QuicConnection / QuicStream public API on top of zquic's embedder I/O surface (feedPacket, processPendingWork, raw application streams, sendRawStreamData, startHandshake). Server TLS identity switches from inline DER to PEM file paths (zquic loads cert/key from disk). Test certs regenerated as P-256 PEM for 127.0.0.1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
zig build test --summary all— 141/141 tests passzig build test-quic— 17/17 QUIC tests pass (handshake, bidi, uni, BCAST+SESS framing)