ITADN

ContentLengthError raised despite all bytes received: paused parser drops on_message_complete when server closes early (3.14 regression)

#13348OpenOdacchi 创建于 22 天前
O
Odacchicommented
## Describe the bug Since aiohttp 3.14.0, a client that consumes a large `Content-Length` response slowly (e.g. application-level bandwidth throttling with `asyncio.sleep` between `iter_chunked()` reads) fails with a contradictory error when the server sends the whole body quickly and then half-closes the connection (as Amazon S3 does): ``` aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header (received 19292859 of 19292859 bytes).'> ``` Note **received == expected** — every byte of the body was received and fed to the parser, yet `ContentLengthError` is raised. ## Root cause analysis (source inspection of 3.14.3) The regression was introduced by the parser pause machinery added in #11966 (3.14.0). Sequence of events: 1. The application consumes the `StreamReader` slowly, so the buffer exceeds the high-water mark and flow control pauses reading: `StreamReader` → `protocol.pause_reading()` → the (C) parser sets `_paused = True`. 2. In `_http_parser.pyx`, `cb_on_body` feeds the final chunk to the payload and then returns `HPE_PAUSED` because `pyparser._paused` is set. llhttp stops **before `on_message_complete` fires**, even though `content_length` has been fully consumed. `self._payload` therefore stays active. 3. The server (S3) already sent the whole body and half-closed; once the transport delivers the close, `client_proto.connection_lost()` calls `parser.feed_eof()`. 4. `feed_eof()` in `_http_parser.pyx` only checks the `F_CONTENT_LENGTH` flag and raises unconditionally. At that point `cparser.content_length == 0`, so the message reads "received N of N bytes". The key asymmetry: the **read-until-EOF** branch of `feed_eof()` has an `_eof_pending` mechanism that defers EOF handling while the parser is paused, but the **content-length** (and chunked) branches do not. The pure-Python parser (`http_parser.py`, `HttpPayloadParser.feed_eof`) has the same asymmetry. 3.13.5 is not affected: its parser has no pause state, so the fully-fed body completes the message immediately. ## To Reproduce Deterministic self-contained reproduction (verified: fails on 3.14.3, completes on 3.13.5). TLS is required — with plain TCP the paused transport defers the EOF, but `SSLProtocol` keeps delivering already-buffered decrypted data and the close while the application-level transport is paused, which is exactly the S3 situation. A large `SO_SNDBUF` lets the server hand the whole body to the kernel and close immediately (early close, like S3). ```bash openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 1 \ -nodes -subj "/CN=127.0.0.1" python repro.py ``` ```python # repro.py — aiohttp 3.14.3: ClientPayloadError/ContentLengthError # aiohttp 3.13.5: prints "done 6000000" import asyncio import pathlib import socket import ssl import aiohttp HERE = pathlib.Path(__file__).parent SIZE = 6_000_000 THROTTLE_BPS = 200_000 # slow consumer -> flow-control pause async def serve(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None: sock = writer.get_extra_info("socket") sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 8 * 1024 * 1024) await reader.readline() writer.write(b"HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n" % SIZE + b"x" * SIZE) await writer.drain() writer.close() # close right after the last byte reaches the kernel, like S3 async def main() -> None: server_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) server_ctx.load_cert_chain(HERE / "cert.pem", HERE / "key.pem") server = await asyncio.start_server(serve, "127.0.0.1", 8443, ssl=server_ctx) async with server: async with aiohttp.ClientSession( connector=aiohttp.TCPConnector(ssl=False) ) as session: async with session.get("https://127.0.0.1:8443/") as resp: n = 0 loop = asyncio.get_running_loop() start = loop.time() async for chunk in resp.content.iter_chunked(64 * 1024): n += len(chunk) expected = n / THROTTLE_BPS elapsed = loop.time() - start if expected > elapsed: await asyncio.sleep(expected - elapsed) print("done", n) asyncio.run(main()) ``` Originally observed downloading ~19 MB objects from S3 via presigned HTTPS URLs with 3 concurrent throttled downloads (41.7 KB/s each): the first download to enter the "server finished sending, client still draining" state failed nearly every cycle. ## Expected behavior The response completes successfully: all bytes were received; a paused parser receiving EOF after the body is fully fed should complete the message (or defer EOF like the read-until-EOF branch does) instead of raising `ContentLengthError`. ## Logs/tracebacks ``` aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header (received 19292859 of 19292859 bytes).'> ``` ## Python Version 3.12 / 3.14 (reproduced on both) ## aiohttp Version 3.14.3 (regression introduced in 3.14.0; 3.13.5 unaffected) ## Related - #11966 (pause machinery introduction, 3.14.0) - #12753 (added received/expected bytes to the error message, which is how the contradictory "received N of N" became visible) - #12953, #12994, #13249 (other 3.14 issues in the same pause/drain machinery) ## OS Linux (Raspberry Pi OS / Debian), also reproduced on macOS ## Additional context Workarounds we found: pin `aiohttp<3.14`, avoid long draining windows (range-split downloads), or raise `read_bufsize` above the object size so flow control never pauses.
1 条评论