Stop output containers blocking the process on a stalled network URL - #2412
Open
adrianrfreedman wants to merge 3 commits into
Open
adrianrfreedman wants to merge 3 commits into
adrianrfreedman wants to merge 3 commits into
Conversation
av_interleaved_write_frame() already released it, but the calls either side held it: avio_open() and avformat_write_header() in start_encoding(), av_write_trailer() and avio_closep() in close_output(). Opening a stream to an unreachable URL stopped every other Python thread. Safe with custom Python I/O, since pyio_read, pyio_write, and pyio_seek are nogil and re-acquire the GIL themselves.
The interrupt callback was only installed for demuxing, so av.open(url, "w", timeout=3.0) ignored the argument and a stalled connect never ended. Install it for both branches and arm it around the open and the header write, as InputContainer does around avformat_open_input(). avio_open() takes no interrupt callback, so start_encoding() now calls avio_open2() and passes the context's own callback down. The callback is disarmed as it is installed, because its deadline starts zeroed and zero reads as already expired. This covers network URLs, not a custom Python file object whose own write() blocks. The callback is only consulted inside FFmpeg.
adrianrfreedman
force-pushed
the
fix/output-blocking-network
branch
from
September 17, 2026 17:48
192e36b to
f4f98d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2400.
Writing to a network URL wedges the whole process.
av.open(url, "w")holds the GIL across the connect and the handshake, so every other Python thread stops, and thetimeoutargument is ignored for output containers, so there is nothing to end the wait. On an unreachable RTMP server the interpreter never comes back.Two separate causes.
The GIL is held across the blocking calls
av_interleaved_write_framealready releases it, so muxing was fine, but the calls either side of it did not:avio_openstart_encoding, connects to the URLavformat_write_headerstart_encoding, writes the header over that connectionav_write_trailerclose_outputavio_closepclose_outputAll four are now wrapped in
with cython.nogil. This is safe for custom Python I/O as well, becausepyio_read,pyio_write, andpyio_seekare alreadynogiland re-acquire the GIL themselves. That is the same reasonav_interleaved_write_framecould be wrapped.timeoutnever reached an output containerThe interrupt callback was only installed on the demuxing branch of
Container.__cinit__, soav.open(url, "w", timeout=3.0)accepted the argument and did nothing with it. It is now installed for both, andstart_encodingarms it around the open and the header write the same wayInputContainerdoes aroundavformat_open_input.Installing it is not enough on its own.
avio_opentakes no interrupt callback, so the protocol got a NULL one and a stalled connect could not be interrupted.start_encodingnow callsavio_open2and passes the format context's callback down, which is what libavformat's ownio_open_defaultdoes.The callback is disarmed at install time. Its deadline lives in a zeroed struct, and a zeroed deadline reads as already expired, so anything blocking between the install and the first
start_timeout()would have aborted immediately.Reproducing
A socket that accepts the connection and then says nothing, so the RTMP handshake never completes. One thread opens the stream, the main thread counts 1 ms sleeps for a second:
timeout=1.0honouredBoth cases are in
tests/test_output_blocking.py. On main the first one does not fail, it hangs, which is the bug.What this does not cover
write()blocks. The interrupt callback is only consulted inside FFmpeg, so it cannot preempt Python code. This fixes network URLs, where FFmpeg owns the socket.timeout.av_interleaved_write_frameon a dead connection blocks the calling thread until TCP gives up, which is the other half of what the reporter saw. Arming the timer per packet is a behaviour change for every existing muxing user, so I have left it out. Happy to do it separately if you want it.avio_open2's options argument is NULL here, asavio_openhad no way to take one either. Passingoptions.ptrthrough it and letting the protocol consume what it recognises, the wayavformat_open_inputdoes, would be a sensible follow-up: protocol options for output URLs currently have no route in and end up in the unused-options warning.