Skip to content

fix cancel stopped seq#4654

Merged
lvhan028 merged 3 commits into
InternLM:mainfrom
RunningLeon:fix-cancel-stop
Jun 10, 2026
Merged

fix cancel stopped seq#4654
lvhan028 merged 3 commits into
InternLM:mainfrom
RunningLeon:fix-cancel-stop

Conversation

@RunningLeon

@RunningLeon RunningLeon commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Fix that cancel signal reset the status of finished seqs inside pt engine

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@RunningLeon RunningLeon marked this pull request as ready for review June 10, 2026 04:16
Copilot AI review requested due to automatic review settings June 10, 2026 04:16
@lvhan028 lvhan028 requested a review from irexyc June 10, 2026 04:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes cancellation behavior around “stopped” sequences in the PyTorch engine and tightens DP wakeup synchronization ordering to avoid races between wakeup and subsequent warmup/forward work.

Changes:

  • Prevent _on_stop_session from cancelling responses belonging to sequences already STOPPED or TO_BE_MIGRATED.
  • Ensure EngineLoop._send_resp prioritizes finish=True outputs over stale “done” (e.g., CANCEL) response states.
  • Move the DP post-wakeup barrier into BaseModelAgent.wakeup() (scoped to the CPU process group) and add coverage for this ordering.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/pytorch/engine/test_model_agent.py Adds a unit test asserting DP wakeup triggers a barrier inside the wakeup RPC (after cache rebuild).
tests/pytorch/engine/test_abort_stopped_seqs.py New tests covering “do not cancel already-stopped/to-be-migrated seqs” and “finish beats stale cancel” behavior.
lmdeploy/pytorch/engine/model_agent/agent.py Relocates post-wakeup DP synchronization to wakeup() using dist_ctx.cpu_group.
lmdeploy/pytorch/engine/engine.py Adjusts stop-session handling to avoid rejecting responses for already-stopped/to-be-migrated sequences.
lmdeploy/pytorch/engine/engine_loop.py Updates response-type selection so out.finish takes precedence over resp.is_done.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@grimoire grimoire left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lvhan028 lvhan028 merged commit 70d2682 into InternLM:main Jun 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants