[earakely-scale]

What

Two pieces so prime-rl can train on AgentEnv bundles with verifiers as released.

serve --masked-as-error. With the flag, serve answers verify for a rollout the environment failed with HTTP 503 and the reason, rather than mask_sample.

• Who it's for: clients that don't read mask_sample.
• Default: off, so Gym and NeMo RL, which read mask_sample from a normal answer, see no change.

tst/verifiers and .github/workflows/verifiers.yml. A contract job that runs vf-eval through verifiers' NeMo Gym taskset against serve.

• Versions: verifiers 0.4.0 and verifiers main.
• Setup: verifiers gets its own venv, since it pins mcp 2 and the framework mcp 1.
• Status: not a required check. It runs on the same triggers as nemo-gym.yml and trl.yml.

Why

verifiers' NeMo Gym taskset, which prime-rl trains through, takes reward from verify and records mask_sample only as a metric. So a rollout whose sandbox never came up trains as reward 0, and counts as 0 in eval means.

A /verify HTTP error goes a different way:

1. The taskset raises in scoring, and verifiers fails the trace.
2. prime-rl's iter_trainable_traces (orchestrator/algo/base.py) skips failed traces.
3. GRPO's score_group builds its baseline from those same traces.

So the rollout is left out of the loss and out of its group's baseline.

I checked this at verifiers 0.4.0, at the commit prime-rl v0.10.0 pins (e888a719) and at main. A verifiers change that drops masked rollouts is proposed upstream in PrimeIntellect-ai/verifiers#3016; this flag doesn't depend on it.

python -m agent_env.training.nemo_gym serve gym-root/resources_servers/gridsim_v1 --port 9100 --masked-as-error

Tests

• tst/verifiers, run locally against verifiers main (6410684):
  • The test writes the server for the unit tier's loopback bundle and runs serve --masked-as-error.
  • It plays two rows through vf-eval nemo-gym (null harness, subprocess runtime, streamed stand-in model).
  • The played row is scored 1.0. The row the server masks (its task doesn't match the episode's) comes back failed with the 503.
  • Without the flag the same test fails: the masked row is a success scored 0.
• The tst/nemo_gym serve test now runs serve --masked-as-error. A played rollout still answers 200 and 1.0, and a masked one answers 503 with the reason.
• tst/nemo_gym, tst/unit/training and the dependency guard pass with nemo-gym installed (48), and in a dev-only venv without it (the verifiers tier skips there with agentenv-capability-missing: nemo_gym_installed).
• On a real bundle (GridSim v1, local Docker), with a stand-in model submitting the idle schedule:
  • 8 eval days through vf-eval all scored 0.20, as through Gym.
  • Two days forced to fail (a 0.01s reset_timeout) came back [error], err 1.00, reward — with the flag. Without it they were [success] with reward 0.00.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; both previous findings are addressed.

Summary

Adds opt-in HTTP 503 responses for masked rollouts and a verifiers contract job.

• Masked rollouts can return HTTP 503 when callers opt in.
• A contract job now runs verifiers through the NeMo Gym server.

_{Reviews (2) · Last reviewed commit: "fix(nemo-gym): serve merges the written ..." · Reviewed by Greptile}