Question: why there will exist two listener FDs

[BamLubi]

Problem:

I'm trying to realize the connection migration related to UDP.
I found that there will exist two FDS on the same UDP port, the simplest test is as follows.

#[tokio::test]
async fn test() {
    let server = Server::builder()
        .with_tls((
            std::path::Path::new("./certificates/www.example.org.cert.pem"),
            std::path::Path::new("./certificates/www.example.org.key.pem"),
        ))
        .unwrap()
        .with_io("127.0.0.1:1443")
        .unwrap()
        .start()
        .unwrap();
    tokio::time::sleep(tokio::time::Duration::from_secs(60)).await;
    drop(server);
}

I wonder to know how to make only one listener FD on single port.

[maddeleine]

Hi BamLubi, I believe that s2n-quic binds twice to the same address so that one file descriptor can be used for reading and one file descriptor can be used for sending (I might be mistaken here, I haven't looked too deeply into how our sockets are set up.)
With that in mind, I'm not sure your request to only have one file descriptor makes sense? We definitely have a very configurable IO, but I'm not quite sure I understand your question yet. Adding more context about your usecase would be helpful here. Also, I should note that s2n-quic does not initiate connection migration. We can respond to a client's request to migrate, but we do not actually initiate any connection migration.

[boquan-fang]

Hi @BamLubi,

We do have an example (test) showing how a s2n-quic server handle connection migration requests from a Cloudflare Quiche client in https://github.com/aws/s2n-quic/blob/main/quic/s2n-quic-tests/src/tests/zero_length_cid_client_connection_migration.rs. You can run

cargo test tests::zero_length_cid_client_connection_migration::zero_length_cid_client_connection_migration_test -- --nocapture

to track down the test log of the server.

[BamLubi]

Thank you for your replies @maddeleine @boquan-fang .

s2n-quic bind twice to the same address?

I am not sure whether s2n_quic should necessary use one file descriptor to read and the other one to write. I have implemented a UDP server using Golang and found that only one FD would be created. Of course, this server can also achieve the sending and receiving of message. The core code is as following.

import (
	"devtool/lib/server"
	"fmt"
)

func UDP_server() {
	addr, err := net.ResolveUDPAddr("udp", ":10001")
	if err != nil {
		fmt.Println("resolve udp addr fail:", err)
		return
	}

	conn, err := net.ListenUDP("udp", addr)
	if err != nil {
		fmt.Println("listen fail:", err)
		return
	}
	defer conn.Close()

	buffer := make([]byte, 1024)
	for {
		n, clientAddr, err := conn.ReadFromUDP(buffer)
		if err != nil {
			fmt.Println("read fail:", err)
			continue
		}

		msg := string(buffer[:n])
		fmt.Printf("receive %s message: %s\n", clientAddr, msg)

		reply := fmt.Sprintf("have receive your message: %s", msg)
		_, err = conn.WriteToUDP([]byte(reply), clientAddr)
		if err != nil {
			fmt.Println("wriet fail:", err)
		}
	}
}

the reason I am focusing on the FDs number

We want to start the s2n-quic server with custom IO Provider, where we will reuse the exist socket. The specific process is, we will first listen to a udp address and then start s2n_quic server based on the achieved socket. In some cases, we could transmit the socket to other Process/Thread and start the s2n_quic server too.

As mentioned in with_rx_socket annotations, if we only set the rx_socket, this socket will also be used as tx_socket. Hence, I make the assumption that where will be only one file descriptor to the adddress. However, the truth is there will exist two FDs, one is what we have specified by socket2::Socket::from_raw_fd(fd), and the other one is new.

I'm not sure if we maintain two FDs to one address can meet the requirements, but i don't think it will be a good method which will increase the cost of FDs management for us.

....
let mut sock = unsafe { socket2::Socket::from_raw_fd(fd) };
let sock: std::net::UdpSocket = sock.into();
....
let io = s2n_quic::provider::io::tokio::Builder::default()
	.with_rx_socket(socket)?
	.build()?;
let server = s2n_quic::Server::builder()
	.with_tls((Path::new("./certs/cert.pem"), Path::new("./certs/key.pem")))?
	.with_io(io)?
	.start()?;

about connection migration

Perhaps I didn't make myself clearly. I'm not referring to the connection migration of QUIC, but share the same sockets between the old process and the new one (by Unix Domain Socket), which make us no need to start listen on the address again (the similar one is TCP connection migration).

[maddeleine]

The tokio::Builder you're using will clone the rx_socket if you did not provide a tx socket. That this is where the second file descriptor is coming from. I don't know why that set up doesn't work for you, but if you do want to customize this you'd have to implement the IO Provider trait. This would be what you would pass to the with_io() call when building a client or server.

[BamLubi]

It means that if clone the rx_socket will generate the second file descriptor, and there will be no way to reuse the same file descriptor for rx_socket and tx_socket, especially because of the ownership of Rust.