Live TS: high-water backpressure cycles healthy live connections, and reconnects request an unsatisfiable byte frontier

[tschuegy]

What happened?

Playing a live MPEG-TS stream (Xtream-style panel, LoadOptions.isLive = true) through the persistent AVIO reader, I expected one long-lived connection. Instead the reader's own backpressure cycles the connection forever, and against some panels the reconnect can never succeed at all:

1. The 16 MB high-water end (Long-suspended data tasks holding full receive windows starve every Network.framework flow in the app process on tvOS — close-and-rerequest at high water would avoid the dormant state (field dose-response inside) #310) has no live branch, and live connections are open-ended by design - so the high-water end is the only thing that ever terminates a healthy live connection. Each end drains ~8 MB to low water and re-requests "at the frontier", but everything broadcast during the drain is gone: the demuxer rejoins on a corrupt TS packet. Panels serve a ring-buffer join burst at line rate on every (re)connect, so the burst refills the window immediately and each reconnect causes the next one - a steady cycle every ~9.5 MB with one mpegts "Packet corrupt" + h264 decode error per cycle, forever.

2. The reconnect request itself carries Range: bytes=<frontier>-, but the frontier is reader bookkeeping, not a server-side byte address. A panel that ignores the offset and serves "from now" masks this; a panel that answers 416 to any offset it cannot satisfy turns every reconnect into an unrecoverable rejection loop: same offset ~1/s (read progress from the draining window keeps resetting the unproductive streak, so it never escalates), re-resolving the pin at streak 2 changes nothing (the offset is the problem, not the pin), and the session starves into CoreMedia -12888.

3. Compounding (1): the 509 a connection-capped panel answers while the replaced connection's slot still lingers server-side is classified as a hard 5xx, so every cycle drops the pinned post-redirect URL and re-resolves through the portal - 500-1500 ms extra gap per cycle, and a second request against exactly the origin that has no room for it (the 519ae26 reasoning, left incomplete for 509).

Steps to reproduce

1. LoadOptions with isLive = true; url = a live .ts from an Xtream-style panel (302 portal redirect to the stream host, max_connections = 1)
2. engine.load(url:options:) and let it play
3. Watch the [AVIOReader] log. Panel type A (ignores Range offsets): "pump window high water: 16MB ahead; ending the connection..." every ~30 s with one "[mpegts] Packet corrupt" per cycle, first reconnect attempt of each cycle 509s and drops the pin. Panel type B (rejects nonzero offsets; every response completes cleanly after a ~14 MB ring burst): "pump gen=N rejected response status=416 at offset X" repeating with the same X until the runway is gone and AVPlayer dies.

AetherEngine version or commit SHA

6.13.0 / 234cb57

Host app

Custom / my own integration

Platform

tvOS

OS version

tvOS 26.6

Device / chip

Apple TV 4K

Playback path

Native AVPlayer

Source media (for playback bugs)

Live MPEG-TS over http. Reproduced on both paths: H.264 High 1080i25 + AAC stereo, SDR (software path) and H.264 Baseline 1080p30 progressive + AAC stereo, SDR (native/loopback path).

Error codes / log lines

[AVIOReader] pump window high water: 16MB ahead; ending the connection, will re-request at the frontier once the consumer drains
[AVIOReader] pump gen=2 rejected response status=509 at offset 17829072
[AVIOReader] Dropped resolved URL cache (hard 509 from pinned URL)
[mpegts @ 0x11b2e8000] Packet corrupt (stream = 0, dts = 3137396400).
[h264 @ 0x12fb95400] error while decoding MB 9 6, bytestream -9
--- panel type B ---
[AVIOReader] pump gen=2 rejected response status=416 at offset 14472616
[AVIOReader] pump replacing a faulted connection at offset 14472616 with 7477KB of read-ahead left (streak=1 status=416, next attempt in 1.0s)
CoreMediaErrorDomain -12888 (Playlist File unchanged for longer than 1.5 * target duration)

Anything else

Root-caused in AVIOReader; a PR fixing all three follows (live high water absorbing the join burst once, live reconnects requesting bytes=0- like a join, 509 joining the 429/503 metering class). Verified on device against both panel types.

[superuser404notfound]

Released in 6.15.2. #332 merged as 1187bd71, plus one follow-up on the reconnect shape in #336 (dad0f00a).

The follow-up is worth knowing about before the retest, because it changes what panel type B's log looks like. #332 made every live request bytes=0-. That is right for a ring-buffer panel, but LoadOptions.isLive is a host declaration that also covers a live source which IS byte-addressable (a growing stream file, a misdeclared VOD): that one answers the frontier correctly and resumes there, and asking it for byte zero re-delivers its whole buffer on top of the window. The flag cannot separate the two shapes, so the origin's own answer does it now. The frontier is kept until a 416 comes back on a nonzero live offset, and that rejection latches the join shape for the rest of the session.

What that means per panel:

• Panel A (ignores Range offsets): no change from what was verified on device. One connection, pumpAheadMB plateauing at burst size, no periodic 509s.

• Panel B (416s nonzero offsets): one rejected request per reader instead of one per reconnect. Expect a single line early in the session, then nothing like it again:

  [AVIOReader] pump live origin rejected offset N (416); requesting the stream as a join from here
  

  A second such line, or a repeat of rejected response status=416 at the same offset, would mean the latch is not holding, and that is worth reopening on.

Every guard was witness-checked here by disabling it: reverting the live high water parks the window at 16.9 MB mid-burst, never latching the join shape starves after one 4 MB burst with 13 rejections at one offset, latching it unconditionally makes a range-honouring live origin's refill restart at byte zero, and taking 509 back out of the rate-limit class grinds 13 attempts with 12 portal re-resolves. Full suite green, 1623 tests.

Leaving this open until the retest on device. Thanks for the diagnosis and the PR, the three defects and the traces that produced them were all in the report already.

[tschuegy]

Retested on device (Apple TV 4K, tvOS 26.6) with 6.15.2 against both panel types from the report:

• Panel type A (ignores Range offsets, answers 509 while the replaced slot lingers): the ~30 s end–drain–reconnect cycle is gone — one connection, pumpAheadMB plateaus at burst size, no periodic Packet corrupt / h264 decode errors, no 509/re-resolve churn.
• Panel type B (cleanly completes each response after its ~14 MB ring burst, 416s nonzero offsets): plays continuously across the burst reconnects; the fix(io): latch the live join shape on rejection instead of assuming it (#331 follow-up) #336 latch behaves as described — a single rejected-offset line early in the session, none after.

Both channel types clean over extended sessions. Thanks for the fast merge and the #336 refinement — closing.