# Week 19 — Update ## TL;DR Improved QUIC interop behavior in `eth-p2p-z`, fixing several edge cases in connection setup and stream handling so that our implementation behaves correctly against other libp2p stacks. PR: [https://github.com/zen-eth/eth-p2p-z/pull/86](https://github.com/zen-eth/eth-p2p-z/pull/86) --- ## Wire behavior - Ensure QUIC handshake and connection parameters match libp2p interop expectations (ALPN, transport-level settings). - Fix mismatches in how streams are opened/accepted so both sides agree on unidirectional vs bidirectional stream usage. - Align close/abort semantics with peers: - Properly send/handle stream reset/close frames. - Avoid leaving half-closed or dangling streams that peers interpret as protocol errors. - Normalize error handling on I/O failures: - Map transport errors to clear disconnect reasons. - Avoid noisy reconnect loops caused by misinterpreting transient QUIC errors. - Respect libp2p’s backoff/limits around connection attempts and concurrent streams to prevent being treated as misbehaving by other implementations. --- ## Per-peer state & invariants For each QUIC peer, maintain a `QuicPeerState`: - `conn_state`: `{ Handshaking, Open, Closing, Closed }`. - `streams_open`: current number of active streams. - `streams_limit`: configured maximum concurrent streams per peer. - `pending_outbound`: queue of pending outbound stream requests when at limit. - `last_error`: last QUIC/transport-level error observed for this connection. - `interop_flags`: - `remote_accepts_bidi`: whether the remote successfully accepted bidirectional streams. - `remote_close_behavior`: observed behavior on our stream close/reset (used to adapt). Invariants: - `streams_open <= streams_limit` at all times. - No events are dispatched to higher layers once `conn_state` is `Closed`. - `pending_outbound` is drained only when `conn_state == Open`. - When `conn_state` transitions to `Closing`, all `streams_open` must either: - Properly complete, or - Receive a well-formed reset/close signal before `Closed`. --- ## Internal APIs - `quic_peer.init(config)` Initialize QUIC transport with interop-safe defaults and limits. - `quic_peer.open_connection(peer_id) -> QuicPeerState` Performs QUIC handshake with correct ALPN/config and sets `conn_state = Open` on success. - `quic_peer.open_stream(peer_id, kind) -> StreamHandle` - Respects `streams_limit`; if at limit, pushes into `pending_outbound`. - Chooses uni/bidi according to interop configuration and observed `interop_flags`. - `quic_peer.handle_incoming_stream(peer_id, stream)` - Classifies stream (uni/bidi, direction). - Hands off to higher-level protocol muxer. - `quic_peer.close_stream(peer_id, stream, reason)` - Sends appropriate close/reset frame. - Decrements `streams_open` and may drain `pending_outbound`. - `quic_peer.on_transport_error(peer_id, error)` - Updates `last_error`. - Decides whether to retry, temporarily back off, or mark peer as incompatible. Configuration knobs: - `max_concurrent_streams` (per peer). - `handshake_timeout`. - `idle_timeout`. - `interop_mode` (e.g. `strict-libp2p` vs `experimental`). --- ## Example pseudocode ```pseudo function open_stream(peer, kind): if peer.conn_state != Open: return error("connection not open") if peer.streams_open >= peer.streams_limit: peer.pending_outbound.push((kind)) return pending stream = quic_conn_open(peer.quic_conn, kind) peer.streams_open += 1 return stream function handle_incoming_stream(peer, stream): if peer.conn_state != Open: reset_stream(stream, reason="closing") return dispatch_to_muxer(peer, stream) function close_stream(peer, stream, reason): send_stream_close_or_reset(stream, reason) peer.streams_open -= 1 if not peer.pending_outbound.is_empty(): kind = peer.pending_outbound.pop() open_stream(peer, kind) function on_transport_error(peer, error): peer.last_error = error peer.conn_state = Closing for s in all_active_streams(peer): reset_stream(s, reason=error) quic_close_connection(peer.quic_conn, error) peer.conn_state = Closed ``` --- ## Message format & transport - Use the existing libp2p QUIC transport framing; no changes to higher-level protocol payloads. - Ensure ALPN and transport parameters match the expected libp2p QUIC profile so other implementations recognize the connection. - Keep stream-level payload formats unchanged; fixes are confined to how streams are created, managed, and closed. - Respect per-message and per-stream limits to avoid triggering defensive behavior on remote peers. --- ## Sequence diagram ```mermaid sequenceDiagram participant A as eth-p2p-z (local) participant B as remote libp2p node A->>B: QUIC handshake (with correct ALPN + params) B-->>A: Handshake OK (connection Open) A->>B: Open bidi stream B-->>A: Accept stream A->>B: Protocol messages over stream B-->>A: Responses over same stream A->>B: Stream close/reset (well-formed) B-->>A: Acknowledges close (no protocol error) A->>B: QUIC connection close B-->>A: Connection teardown complete ``` --- ## Next - Add targeted interop tests against other libp2p QUIC implementations (go-libp2p, rust-libp2p). - Wire transport-level metrics (stream usage, error rates, handshake failures) into monitoring. - Use these signals in the peer manager to refine connection scoring and backoff logic. - Continue hardening the QUIC implementation under adversarial and high-load scenarios. --- ## References - PR: https://github.com/zen-eth/eth-p2p-z/pull/86