Subsequent requests on reused connection delayed by 40 milliseconds
When running oxhttp with a client that supports connection reuse on Linux, responses after the first response on the same connection experience a 40 millisecond delay:
$ time curl -s http://localhost:8080 http://localhost:8080 > /dev/null
real 0m0.045s
This delay occurs due to an interaction between oxhttp, Nagle's algorithm, and TCP delayed ACK:
* oxhttp writes the response headers and body to the socket separately, resulting in two separate TCP segments.
* [TCP delayed ACK](https://en.wikipedia.org/wiki/TCP_delayed_acknowledgment) causes the client to wait before acknowledging the first segment, because it is smaller than the maximum segment size.
* [Nagle's algorithm](https://en.wikipedia.org/wiki/Nagle%27s_algorithm) causes the server to wait before sending the second segment, because the first segment is unacknowledged.
I think the first response on a connection is unaffected because [Linux always acknowledges the first data segment immediately](https://github.com/torvalds/linux/blob/fac04efc5c793dccbd07e2d59af9f90b7fc0dca4/net/ipv4/tcp_input.c#L829-L832).
This could be fixed in oxhttp by either:
* Writing the response headers and body to the socket together.
* Disabling Nagle's algorithm using [`TcpStream::set_nodelay`](https://doc.rust-lang.org/std/net/struct.TcpStream.html#method.set_nodelay).
6 条评论