fix(suite): stop suites stalling and agents spinning - #241
Open
Centaurus99 wants to merge 7 commits into
Open
Conversation
Decrement inside the archiving transaction, increment by column expression, and fold the cancelled-suite check into that statement.
Centaurus99
force-pushed
the
suite-task-reclaim
branch
from
August 16, 2026 03:37
c821e26 to
c383fb1
Compare
Offer a suite only while it holds a Ready task, and skip the immediate re-accept after a job that ran none.
Centaurus99
changed the title
Centaurus99
changed the title
Comment on lines
+333
to
+347
| Some(suite) => Some( | ||
| accept_task_into_suite_query(suite.id, now) | ||
| .exec_with_returning(txn) | ||
| .await? | ||
| .into_iter() | ||
| .next() | ||
| .ok_or_else(|| { | ||
| Error::ApiError(crate::error::ApiError::InvalidRequest( | ||
| format!( | ||
| "Suite {} was cancelled and cannot accept new tasks", | ||
| suite.uuid | ||
| ), | ||
| )) | ||
| })?, | ||
| ), |
There was a problem hiding this comment.
I think we should revert this accept_task_into_suite_query back to the original orm form, as the state of the suite is already checked in suite.state.can_accept_tasks() and using update_many here does not save us db connection RTT.
Member
|
@un-lock-able You should check what we talked about in feishu and reimplement this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three ways a suite could stall, or keep agents busy doing nothing:
incomplete_taskslost its decrements. Submission wrote backprev + 1read from an unlocked row, overwriting anyincomplete_tasks - 1a commit landed in between. A 4974-task suite settled at 4476 with every task finished, so-Wnever returned. Both counters now move by column expression, and the decrement moves inside the archiving transaction on both commit paths.Uncommitted tasks were stranded. An agent completing its job while holding
Running/Finished/Cancelledtasks left them inactive_tasksforever, unclaimable by anyone.agent_complete_jobnow reclaims them back toReadyfor a re-run, dropping that run's results.Agents spun on a suite with nothing to claim.
suite_has_workasked forincomplete_tasks > 0, which also counts tasks running elsewhere; one suite accumulated 25k accept/complete jobs. It now requires aReadytask, and an agent whose job ran no task no longer re-accepts immediately.🤖 Generated with Claude Code