fix(async): honor max_num_epochs in async GRPO

[bzantium]

What does this PR do ?

async_grpo_train bounds itself with while step < max_num_steps and has no epoch loop, since it consumes the trajectory collector's buffer rather than iterating the dataloader. grpo.max_num_epochs therefore had no effect on an async run: a config asking for one epoch kept going until the collector reported an exhausted dataloader and raised.

setup() already derives the epoch-equivalent step count for the Megatron scheduler budget. This clamps max_num_steps with it, which bounds the loop and keeps the scheduler horizon in agreement with the loop it schedules for. Multiple dataloaders are exempt, since MultipleDataloaderWrapper is an infinite iterator and an epoch has no length to derive a bound from.

The exhaustion message also pointed at data.train.max_num_epochs, which nothing reads; the setting is grpo.max_num_epochs.

Issues

None closed.

Usage

grpo:
  max_num_epochs: 1
  max_num_steps: 1000000   # upper bound; the epoch clamp decides where the run ends
  async_grpo:
    enabled: true

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally?
• Did you add or update any necessary documentation?

Additional Information

• tests/unit/algorithms/test_grpo_max_num_steps.py covers the clamp, a multi-epoch budget, an already-smaller step budget, the multiple-dataloader exemption and a non-positive epoch count.
• The five cases were verified against the helper in isolation, but the test module was not run under pytest in a full environment, so that box is left unchecked.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.