Add multimodal preprocessing metrics#4640
Open
CUHKSZzxy wants to merge 7 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end metrics for multimodal (VLM) prompt preprocessing, wiring new stats collection from request processing through the existing metrics processor and Prometheus/console loggers, and documenting the exported metrics.
Changes:
- Introduces
MultimodalStatsto track multimodal item counts, per-stage/total preprocessing latency, and failures. - Instruments
MultimodalProcessorandAsyncEngine.generate()to collect and emit multimodal preprocessing stats viametrics_processor. - Extends metrics loggers (Prometheus + periodic console logger) and updates EN/ZH metrics documentation; adds targeted tests for the new multimodal metrics.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_lmdeploy/test_metrics_multimodal.py | Adds unit tests covering multimodal stats behavior and Prometheus emission. |
| lmdeploy/serve/processors/multimodal.py | Instruments multimodal parsing + VLM preprocessing stages and threadsafe stats updates. |
| lmdeploy/serve/core/async_engine.py | Creates/records per-request multimodal stats in the request path. |
| lmdeploy/metrics/stats.py | Adds MultimodalStats for multimodal preprocessing accounting. |
| lmdeploy/metrics/metrics_processor.py | Adds record_multimodal() to emit multimodal stats through configured loggers. |
| lmdeploy/metrics/loggers.py | Implements multimodal metric export for Prometheus and aggregates for console logging. |
| docs/zh_cn/advance/metrics.md | Documents newly exported multimodal preprocessing metrics (CN). |
| docs/en/advance/metrics.md | Documents newly exported multimodal preprocessing metrics (EN). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| mm_stats.record_failure('media_io', 'unknown') | ||
| raise NotImplementedError(f'unknown type: {item_type}') | ||
|
|
||
| out_message['content'].append({'type': modality.value, 'data': data, **item_params}) |
Collaborator
Author
There was a problem hiding this comment.
This is existing behavior from main: the stored value is a string, and Modality.eq supports comparing against that string.
# Conflicts: # lmdeploy/serve/processors/multimodal.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
Benchmark
Terminal Log
When a logging interval includes multimodal preprocessing, the terminal stats line now appends the average multimodal preprocessing latency:
Assistance
Assisted with Codex + GPT-5.5 xHigh Fast