Run async UDFs on a dedicated event-loop thread

[nkitsaini]

Note

Medium Risk
Changes core UDF invoke concurrency and cancellation semantics on a shared global loop; mitigated by extensive new tests but still affects production request handling under load.

Overview
Moves async UDF execution off the uvicorn/ASGI request loop onto a singleton daemon thread with its own event loop, so heavy or blocking async work is less likely to prevent the server from accepting new connections. Concurrent async invocations are scheduled on that loop via run_coroutine_threadsafe (with contextvars copied across the thread boundary); sync UDFs still run in a worker thread with a per-call asyncio.run so they cannot block the shared dispatch loop.

Adds _cancellable_run to honor cancel_event for work on the dispatch loop (polling because request-loop task cancel does not reach it), prefers a completed result when cancel races with success, and tightens invoke-path cancellation (cancel_event.set() when the func task is still pending and in finally).

Managed UDF uvicorn startup now sets timeout_keep_alive to 120s by default (override via SINGLESTOREDB_UDF_KEEPALIVE_TIMEOUT, with invalid values warned and ignored). New test_udf_event_loop.py covers cancellation, dispatch concurrency, routing, and Application integration.

^{Reviewed by Cursor Bugbot for commit 77c75a6. Bugbot is set up for automated code reviews on this repo. Configure here.}

We have seen that UDFs can block the event loop and stop uvicorn from accepting new connections. In this PR I have moved the execution of async UDFs to a dedicated daemon thread.

I have tested this in preview by rolling out the Nova Image.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 018db7a. Configure here.}

[mgiannakopoulos]

🚢 Ship IT!

[Copilot]

Pull request overview

This PR moves async UDF execution off the request event loop and onto a process-wide dedicated event-loop thread to improve concurrency and allow reuse of loop-bound async resources across requests (e.g., HTTP client pools). It also updates cancellation behavior to work across threads and increases the default uvicorn keep-alive timeout for managed Python UDF servers to reduce connection churn under load.

Changes:

• Add a singleton dedicated async-dispatch event loop running in a daemon thread, and route async UDF invocations through it.
• Implement cross-thread cancellation via a threading.Event polled from the dispatch loop, and wire cancellation signaling into the invoke path.
• Increase managed UDF server keep-alive timeout via SINGLESTOREDB_UDF_KEEPALIVE_TIMEOUT and add tests covering dispatch concurrency/routing/cancellation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
singlestoredb/functions/ext/asgi.py	Introduces the shared async-dispatch loop/thread and updates the UDF invoke path to dispatch async UDFs there with cross-thread cancellation.
singlestoredb/apps/_python_udfs.py	Wires a configurable uvicorn timeout_keep_alive with a higher default to reduce idle connection churn.
singlestoredb/tests/test_udf_event_loop.py	Adds unit/integration tests for dispatch loop behavior, cancellation helper, and Application routing between async/sync UDF execution paths.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kesmit13]

I had Copilot do a review. Can you just check over the comments and see if there is anything that still needs addressing? If it looks ok, I'll merge it.

[nkitsaini]

@kesmit13 one was good catch. rest were improvements.

Good to go now.