From eba59603fb4da0d813b26683af6e641117f54189 Mon Sep 17 00:00:00 2001 From: VizzleTF Date: Sun, 26 Jul 2026 17:00:18 +0300 Subject: [PATCH] client: set TCP_NODELAY on accepted sockets uh_accept_client() leaves the accepted socket with Nagle enabled, and a response is emitted as one write() per header line plus one for the body. The kernel therefore holds the small final segment: tcp_nagle_check() allows a partial segment only if TCP_NODELAY is set, TCP_CORK is set, or every small packet sent so far has been acknowledged (Minshall's modification, tcp_minshall_check()). On a connection that has just answered a request, the previous response is still unacknowledged, so the next small body waits for the peer's delayed ACK. Every request after the first on a keep-alive connection pays that. Reproduced without LuCI, ubus, a session or a browser - a 5-byte static file and four sequential requests on one connection, on 25.12 (x86_64) and 24.10.7 (uhttpd 2025.07.06~7e64e8ba-r4): req1 0.4 ms / 1.0 ms (first on the connection) req2 41.5 ms / 42.2 ms req3 44.6 ms / 48.8 ms req4 44.6 ms / - fresh connection each time: 0.3-0.4 ms / 0.24-0.33 ms strace shows uhttpd is not the slow part: it wrote response 2 within 0.4 ms of response 1, while the client's next request only arrived 42.5 ms after that write. The stall window is the peer's delayed ACK timer, so it depends on the client; the figures above are Linux peers. Both builds come from the 25.12.5 release SDK, and the unpatched build is byte-identical to the published uhttpd-2026.06.16~7b1bec45-r1.apk, so the before column is what is shipping. Same box, http_keepalive at its default: unpatched 0.35 / 42.9 / 44.8 / 44.8 ms patched 0.32 / 0.087 / 0.075 / 0.075 ms LuCI is where this is visible, since its ubus JSON-RPC replies run a few hundred bytes and a page issues several: warm in-place navigation over five config pages went from 540 ms to 284 ms with keep-alive untouched. The trade-off is packet count, and it is measured rather than assumed. Disabling Nagle lets each of the eight writes leave as its own segment: the four-request reproducer goes from 13 to 32 server-to-client segments, and a full LuCI page load from 752 to 1211 packets for the same payload. For an admin interface that is a few dozen extra packets per page view against 40 ms stalls, but the real answer is to coalesce the response into fewer writes as well - nginx does both, enabling TCP_NODELAY when a connection enters the keep-alive state. Worth noting for that: ustream already carries a `more` flag from ustream_write() down to the backend, and ustream_fd_write() ignores it, using write() rather than send() with MSG_MORE - while ustream_vprintf() hardcodes more=false, so no printf-based header emission can express it today. Signed-off-by: VizzleTF --- client.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/client.c b/client.c index 1fc1126..439c81d 100644 --- a/client.c +++ b/client.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "uhttpd.h" #include "tls.h" @@ -823,6 +824,7 @@ bool uh_accept_client(int fd, bool tls) static struct client *next_client; struct client *cl; unsigned int sl; + int yes = 1; int sfd; static int client_id = 0; struct sockaddr_in6 addr; @@ -837,6 +839,13 @@ bool uh_accept_client(int fd, bool tls) if (sfd < 0) return false; + /* A response is emitted as one write() per header line plus one for the + * body, so with Nagle enabled the small final segment of a response on a + * REUSED keep-alive connection is held back until the peer acknowledges + * the previous one - which its delayed ACK timer puts ~40 ms away. Every + * request after the first on a connection pays that. */ + setsockopt(sfd, IPPROTO_TCP, TCP_NODELAY, &yes, sizeof(yes)); + set_addr(&cl->peer_addr, &addr); sl = sizeof(addr); getsockname(sfd, (struct sockaddr *) &addr, &sl);