[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests by xinhe-nv · Pull Request #19178 · NVIDIA/TensorRT-LLM

xinhe-nv · 2026-09-15T02:02:21Z

llama model tests are deprecated. Swaps the TinyLlama-1.1B-Chat-v1.0 test model for Qwen3-0.6B across unit and integration tests, and prunes the now-redundant tinyllama-1.1b decoder-encode parametrization in favor of the existing qwen3-0.6b case.

Kept TinyLlama-1.1B-Chat-v1.0 where swapping would silently break the test: LoRA adapters fine-tuned specifically for TinyLlama (multilora example, triton_server LoRA E2E test), a NeMo LoRA GQA test with hardcoded checkpoint dimensions matching TinyLlama's architecture, and a disaggregated test asserting exact hardcoded token IDs/text produced by TinyLlama's tokenizer.

Dev Engineer Review

Replaces TinyLlama with Qwen3-0.6B across applicable unit and integration tests.
Retains TinyLlama for LoRA adapters, fixed checkpoint dimensions, and exact tokenizer outputs.
Removes deprecated DeepSeek test selections and one obsolete cancellation test.
Adds dedicated TinyLlama LoRA fixtures and Triton coverage.
Test results and review finding counts are unavailable.

QA Engineer Review

Updates unit tests, integration tests, CI test lists, QA lists, model mappings, and waivers.
Adds Llama 3.2 1B decoder accuracy coverage.
Removes redundant Qwen3 parametrization.
Adds Qwen3 model-path validation before affected integration setup.
Retains TinyLlama-specific LoRA coverage.
Coverage verdict: needs follow-up because test execution results are unavailable.

Per-File QA Perspective

Modified test files cover inference, scheduling, disaggregation, serving, telemetry, profiling, sampling, KV cache, and model loading with Qwen3-0.6B.
tests/integration/defs/triton_server/conftest.py adds separate TinyLlama LoRA fixtures.
tests/integration/defs/triton_server/test_triton_llmapi.py adds TinyLlama LoRA request coverage.
tests/integration/defs/disaggregated/test_disaggregated.py removes the large cancellation test and changes model-specific cases to Qwen3.
tests/integration/defs/conftest.py and tests/test_common/llm_data.py remove the general TinyLlama mapping.
tests/integration/test_lists/test-db/*.yml, tests/integration/test_lists/qa/llm_function_core.txt, and tests/integration/test_lists/waives.txt update selectors, coverage, and waivers.
QA should verify that retained TinyLlama tests use the new LoRA fixtures and that Qwen3 tests run on the intended hardware.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Swaps the TinyLlama-1.1B-Chat-v1.0 test model for Qwen3-0.6B across unit and integration tests, and prunes the now-redundant tinyllama-1.1b decoder-encode parametrization in favor of the existing qwen3-0.6b case. Kept TinyLlama-1.1B-Chat-v1.0 where swapping would silently break the test: LoRA adapters fine-tuned specifically for TinyLlama (multilora example, triton_server LoRA E2E test), a NeMo LoRA GQA test with hardcoded checkpoint dimensions matching TinyLlama's architecture, and a disaggregated test asserting exact hardcoded token IDs/text produced by TinyLlama's tokenizer. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…-adapter-locked TinyLlama test Gives Qwen3-0.6B its own plain conftest fixture instead of sharing the request.param-dispatched llama_model_root fixture, and drops the now-unnecessary indirect parametrize boilerplate at every call site since there is only one Qwen3-0.6B variant. Updates the matching test-list entries for the resulting simplified test IDs. Also removes test_llmapi_lora and its tiny_llama_lora_model_root / tiny_llama_lora_base_model_root fixtures: the LoRA adapter under test is fine-tuned specifically for TinyLlama-1.1B-Chat-v1.0 and has no Qwen3-0.6B equivalent, so the test can't be migrated. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Renames the remaining tinyllama-labeled helpers now that the model under test is Qwen3-0.6B: the reduced_layer_handoff parametrize id and its supporting reduced_tinyllama_config()/REDUCED_TINYLLAMA_LAYERS in test_ad_disagg.py. Also removes the triton_server qwen_model_root (formerly tiny_llama_model_root) fixture. It was only used by two tests in the same module, so it's inlined as a plain helper function there instead of adding another module-scoped fixture. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Drops the single-use _qwen_model_root() helper and inlines the os.path.join(models_root, "Qwen3", "Qwen3-0.6B") lookup directly at each of its two call sites. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…okup Drops the qwen_model_root fixture from conftest.py and inlines os.path.join(llm_models_root(), "Qwen3", "Qwen3-0.6B") directly at each of its ~38 call sites across the disaggregated and e2e test suites, matching the same pattern already used for the triton_server tests. Also fixes test_trtllm_bench_invalid_token_pytorch in test_e2e.py, which was routing "Qwen3-0.6B" through the llama_model_root fixture indirect-parametrize (a leftover from before qwen_model_root was split out of llama_model_root, and broken since llama_model_root no longer has a Qwen3-0.6B branch) instead of using its own lookup. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…trtllm-15117 Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…DELS All Llama-family checkpoints, including Llama-3.2-1B, have been retired from this suite. DECODER_MODELS in test_llm_api_pytorch_encode.py is documented as one representative per distinct TRT-LLM architecture class; keeping a LlamaForCausalLM entry pointed at a retired checkpoint would silently break once that checkpoint is gone. Drop the entry and reword the comment so it no longer advertises Llama/Mistral coverage that the list doesn't provide, and move the corresponding test-list entries over to the surviving qwen3-0.6b param. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-09-21T05:39:20Z

/bot run

tensorrt-cicd · 2026-09-21T05:46:02Z

PR_Github #74741 [ run ] triggered by Bot. Commit: db29e56 Link to invocation

tensorrt-cicd · 2026-09-21T07:12:01Z

PR_Github #74741 [ run ] completed with state SUCCESS. Commit: db29e56
/LLM/main/L0_MergeRequest_PR pipeline #61523 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Byte-level BPE tokenizers such as Qwen3's encode a word differently depending on whether it's preceded by a space: 'Paris' and ' Paris' are distinct tokens (59604 and 12095 respectively, verified against Qwen3-0.6B's tokenizer.json). The model generates the space-prefixed variant after "is", so biasing only the bare-word token had no effect on what was actually generated, causing test_completion_with_logit_bias_effect to fail with the negative-bias assertion once the test model moved to Qwen3-0.6B. Bias both variants, and update the stale fallback token ids (3681, from the pre-migration model) to Qwen3's actual ids. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…put check run_client_tests' default output-verification branch asserted the literal string "Asyncio is a Python library", which held for the model previously used on this path but not for Qwen3-0.6B: it answers the raw asyncio prompt with a reasoning-style continuation rather than repeating it verbatim, so tests like test_disaggregated_kv_cache_time_output and test_disaggregated_load_balance failed after the TinyLlama -> Qwen3-0.6B migration. Accept the model's actual phrasing alongside the original expectation, matching the existing pattern used for the deepseek_v3_lite/gpt_oss_120b/qwen3_32b_fp8 branches. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

…ng unconditionally test_disaggregated_logprobs_serving's streaming-vs-non-streaming logprob comparison is already waived under nvbugs/6275959 for the full:A100/B200/B300/H20/L40S QA tier, but not for the base (L0) test-db lists, where it still runs unwaived. Confirmed the failure is a genuine, longstanding cross-model/cross-hardware issue (reproduced on H20 with Qwen3-8B during a debug run, and separately reported on B200 with the pre-migration llama-3.1-8b-instruct parametrization) in the disaggregated streaming-logprob path, not something introduced by the TinyLlama -> Qwen3 test migration on this branch. Add an unprefixed waiver so it's skipped everywhere until nvbugs/6275959 is resolved. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com> # Conflicts: # tests/integration/test_lists/waives.txt

xinhe-nv · 2026-09-22T01:31:57Z

/bot run

tensorrt-cicd · 2026-09-22T01:41:17Z

PR_Github #74911 [ run ] triggered by Bot. Commit: c176d24 Link to invocation

tensorrt-cicd · 2026-09-22T03:55:58Z

PR_Github #74911 [ run ] completed with state SUCCESS. Commit: c176d24
/LLM/main/L0_MergeRequest_PR pipeline #61678 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

… count check_generated_output asserted context_logits.shape[0] against len(prompts[0].split()) + 1, hardcoding the BOS + 3-token split that "A B C" happened to produce under a specific tokenizer. This breaks for any tokenizer where "A B C" doesn't tokenize to exactly one BOS token plus one token per word. Compare against the actual output.prompt_token_ids length instead, which holds regardless of tokenizer. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-09-22T06:21:44Z

/bot run

tensorrt-cicd · 2026-09-22T06:29:27Z

PR_Github #74977 [ run ] triggered by Bot. Commit: 3cf4603 Link to invocation

tensorrt-cicd · 2026-09-22T08:33:54Z

PR_Github #74977 [ run ] completed with state SUCCESS. Commit: 3cf4603
/LLM/main/L0_MergeRequest_PR pipeline #61738 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…s.yaml test_disaggregated_kv_cache_time_output OOMs on A10 (22.30 GiB) during executor creation: "Additional executor resources (temporary for KV cache size estimation)" fails to allocate after model load (13.81 GiB free) and initial KV cache estimation (10.36 GiB free). Unlike sibling configs in this directory (e.g. disagg_config_load_balance.yaml), this config never capped max_batch_size/max_num_tokens/max_seq_len, so the KV-cache-transceiver buffer sizing fell back to large defaults. This went unnoticed under the previous TinyLlama-1.1B test model; Qwen3-0.6B has a much larger vocabulary (151k vs ~32k tokens), which appears to push the uncapped buffer estimate over the edge on constrained hardware. Cap both context_servers and generation_servers to modest values sufficient for this test's short default prompts. Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com> # Conflicts: # tests/integration/test_lists/waives.txt

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv · 2026-09-22T10:59:39Z

/bot run

tensorrt-cicd · 2026-09-22T11:07:16Z

PR_Github #75028 [ run ] triggered by Bot. Commit: 6d88498 Link to invocation

tensorrt-cicd · 2026-09-22T13:25:53Z

PR_Github #75028 [ run ] completed with state SUCCESS. Commit: 6d88498
/LLM/main/L0_MergeRequest_PR pipeline #61783 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

github-actions Bot assigned xinhe-nv Sep 15, 2026

xinhe-nv added 5 commits September 15, 2026 10:33

Merge branch 'main' into trtllm-15117

9f6cea3

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv marked this pull request as ready for review September 15, 2026 08:51

xinhe-nv requested review from a team as code owners September 15, 2026 08:51

xinhe-nv requested review from BowenFu, LarryXFly, Mgluhovskoi, MrGeva, asfiyab-nvidia, brnguyen2, dpitman-nvda, nv-xtf, reasonsolo, tongyuantongyu and yiqingy0 September 15, 2026 08:51

xinhe-nv added 5 commits September 18, 2026 10:11

Merge branch 'main' into trtllm-15117

3efbb84

Merge branch 'main' into trtllm-15117

54dd7dc

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Merge branch 'trtllm-15117' of github.com:xinhe-nv/TensorRT-LLM into …

edcc069

…trtllm-15117 Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Merge remote-tracking branch 'upstream/main' into trtllm-15117

db29e56

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

xinhe-nv added 5 commits September 21, 2026 19:47

Merge remote-tracking branch 'upstream/main' into trtllm-15117

37463e3

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com> # Conflicts: # tests/integration/test_lists/waives.txt

Merge branch 'main' into trtllm-15117

c176d24

xinhe-nv enabled auto-merge (squash) September 22, 2026 01:31

xinhe-nv added 3 commits September 22, 2026 13:30

Merge remote-tracking branch 'upstream/main' into trtllm-15117

05357ca

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Merge remote-tracking branch 'origin/trtllm-15117' into trtllm-15117

4406163

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Merge branch 'main' into trtllm-15117

3cf4603

xinhe-nv added 3 commits September 22, 2026 18:43

Merge remote-tracking branch 'upstream/main' into trtllm-15117

28ea61d

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com> # Conflicts: # tests/integration/test_lists/waives.txt

Merge remote-tracking branch 'origin/trtllm-15117' into trtllm-15117

6d88498

Signed-off-by: Xin He (SW-GPU) <200704525+xinhe-nv@users.noreply.github.com>

Navigation Menu

[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests - #19178

[None][test] Replace TinyLlama-1.1B-Chat-v1.0 with Qwen3-0.6B in tests#19178
xinhe-nv wants to merge 35 commits into
NVIDIA:mainfrom
xinhe-nv:trtllm-15117

xinhe-nv commented Sep 15, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

xinhe-nv commented Sep 21, 2026

Uh oh!

tensorrt-cicd commented Sep 21, 2026

Uh oh!

tensorrt-cicd commented Sep 21, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

xinhe-nv commented Sep 15, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

xinhe-nv commented Sep 21, 2026

Uh oh!

tensorrt-cicd commented Sep 21, 2026

Uh oh!

tensorrt-cicd commented Sep 21, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

xinhe-nv commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

xinhe-nv commented Sep 15, 2026 •

edited by coderabbitai Bot

Loading