Skip to content

[MLPerf 6.1][GRPO] Update Qwen3.5 reference assets - #904

Open
jepio wants to merge 1 commit into
mlcommons:masterfrom
jepio:docs/qwen35-storage-dataset
Open

[MLPerf 6.1][GRPO] Update Qwen3.5 reference assets#904
jepio wants to merge 1 commit into
mlcommons:masterfrom
jepio:docs/qwen35-storage-dataset

Conversation

@jepio

@jepio jepio commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

During dataset publishing to MLCommons storage we identified entries that had different behavior between architectures or were fundamentally broken. The result is that the number of datasets in the training and validation set slightly changed, without impacting convergence behavior of the benchmark.

We are also proposing a change to the reference configuration to reduce command and test timeouts from 300/180 to 60/60. These don't affect correctness/convergence but prevent training stalls and allow for better submissions.

Detailed changes in this PR are:

  • Documents MLCommons storage as the primary source for the qualified Qwen3.5 checkpoint, dataset, and SIF images.
  • Keeps the source preparation and build instructions as optional alternatives.
  • Updates the qualified dataset identities, task counts, runtime settings, and NeMo-RL submodule revision.
  • Links the NeMo-RL updated mlperf-training-qwen35-main branch.

Related compliance checker update:

@jepio
jepio requested review from a team as code owners September 3, 2026 11:26
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

Signed-off-by: Jeremi Piotrowski <jpiotrowski@nvidia.com>
@jepio
jepio force-pushed the docs/qwen35-storage-dataset branch from d4622d4 to 49cba8f Compare September 3, 2026 14:07
Comment thread llm_post_training/RL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the RL version different?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • dataset preparation scripts have been updated
  • documentation has been updated
  • test/command timeout update in the configuration

this shows the diff: NVIDIA-NeMo/RL@3fca04c...644f370

@RissyRan

RissyRan commented Sep 4, 2026

Copy link
Copy Markdown

Thanks for the change and proposal! Have 2 questions:

  1. Similar one about 251 samples in validation in Update Qwen3.5 validation sample count logging#475
  2. What're are actual train and validation test/command timeout in reference runs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants