Skip to content

Stop output containers blocking the process on a stalled network URL - #2412

Open
adrianrfreedman wants to merge 3 commits into
PyAV-Org:mainfrom
adrianrfreedman:fix/output-blocking-network
Open

adrianrfreedman wants to merge 3 commits into
PyAV-Org:mainfrom
adrianrfreedman:fix/output-blocking-network

Conversation

@adrianrfreedman

Copy link
Copy Markdown

Fixes #2400.

Writing to a network URL wedges the whole process. av.open(url, "w") holds the GIL across the connect and the handshake, so every other Python thread stops, and the timeout argument is ignored for output containers, so there is nothing to end the wait. On an unreachable RTMP server the interpreter never comes back.

Two separate causes.

The GIL is held across the blocking calls

av_interleaved_write_frame already releases it, so muxing was fine, but the calls either side of it did not:

call where
avio_open start_encoding, connects to the URL
avformat_write_header start_encoding, writes the header over that connection
av_write_trailer close_output
avio_closep close_output

All four are now wrapped in with cython.nogil. This is safe for custom Python I/O as well, because pyio_read, pyio_write, and pyio_seek are already nogil and re-acquire the GIL themselves. That is the same reason av_interleaved_write_frame could be wrapped.

timeout never reached an output container

The interrupt callback was only installed on the demuxing branch of Container.__cinit__, so av.open(url, "w", timeout=3.0) accepted the argument and did nothing with it. It is now installed for both, and start_encoding arms it around the open and the header write the same way InputContainer does around avformat_open_input.

Installing it is not enough on its own. avio_open takes no interrupt callback, so the protocol got a NULL one and a stalled connect could not be interrupted. start_encoding now calls avio_open2 and passes the format context's callback down, which is what libavformat's own io_open_default does.

The callback is disarmed at install time. Its deadline lives in a zeroed struct, and a zeroed deadline reads as already expired, so anything blocking between the install and the first start_timeout() would have aborted immediately.

Reproducing

A socket that accepts the connection and then says nothing, so the RTMP handshake never completes. One thread opens the stream, the main thread counts 1 ms sleeps for a second:

main-thread wakeups timeout=1.0 honoured
main 0, the process hangs for ever no
this PR ~1200 yes, raises after ~1 s

Both cases are in tests/test_output_blocking.py. On main the first one does not fail, it hangs, which is the bug.

What this does not cover

  • A custom Python file object whose own write() blocks. The interrupt callback is only consulted inside FFmpeg, so it cannot preempt Python code. This fixes network URLs, where FFmpeg owns the socket.
  • Muxing and closing still ignore timeout. av_interleaved_write_frame on a dead connection blocks the calling thread until TCP gives up, which is the other half of what the reporter saw. Arming the timer per packet is a behaviour change for every existing muxing user, so I have left it out. Happy to do it separately if you want it.
  • avio_open2's options argument is NULL here, as avio_open had no way to take one either. Passing options.ptr through it and letting the protocol consume what it recognises, the way avformat_open_input does, would be a sensible follow-up: protocol options for output URLs currently have no route in and end up in the unused-options warning.

av_interleaved_write_frame() already released it, but the calls either side
held it: avio_open() and avformat_write_header() in start_encoding(),
av_write_trailer() and avio_closep() in close_output(). Opening a stream to
an unreachable URL stopped every other Python thread.

Safe with custom Python I/O, since pyio_read, pyio_write, and pyio_seek are
nogil and re-acquire the GIL themselves.
The interrupt callback was only installed for demuxing, so
av.open(url, "w", timeout=3.0) ignored the argument and a stalled connect
never ended. Install it for both branches and arm it around the open and the
header write, as InputContainer does around avformat_open_input().

avio_open() takes no interrupt callback, so start_encoding() now calls
avio_open2() and passes the context's own callback down.

The callback is disarmed as it is installed, because its deadline starts
zeroed and zero reads as already expired.

This covers network URLs, not a custom Python file object whose own write()
blocks. The callback is only consulted inside FFmpeg.
@adrianrfreedman
adrianrfreedman force-pushed the fix/output-blocking-network branch from 192e36b to f4f98d0 Compare September 17, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

push rtmp stream block the main thread

1 participant