fix: consolidate Megatron Inference tests

[tdene]

What does this PR do ?

PER-PR FUNCTIONALS  (Qwen2.5-0.5B / Qwen3-0.6B, 2 GPUs; "fast" also runs in the FAST/Lfast lane)
Runs on both the H100 and GB200 L1 runners.
+----------------------------+------+---------------------+-----+-------------------------------+
| L1 Megatron_4 shard        | fast | deployment          | lag | gates                         |
| (grpo_megatron_generation_)|      |                     |     |                               |
+----------------------------+------+---------------------+-----+-------------------------------+
| colocated                  |      | colocated           |  0  | mult_prob_error max < 1.05    |
| non_colocated              |      | non-colocated       |  0  | mult_prob_error max < 1.05    |
| colocated_reshard          |      | colocated, reshard  |  0  | mult_prob_error max < 1.05    |
| colocated_gym              |      | colocated           |  0  | mult_prob_error max < 1.05    |
|                            |      |                     |     | gen_kl_error median < 1.3     |
|                            |      |                     |     | accuracy > 0.1                |
| non_colocated_gym          |      | non-colocated       |  0  | mult_prob_error max < 1.05    |
|                            |      |                     |     | gen_kl_error median < 1.3     |
|                            |      |                     |     | accuracy > 0.1                |
| topp_topk                  | fast | colocated, top-p/k  |  0  | mult_prob_error max < 1.10    |
| non_colocated_async_gym    | fast | non-colocated       |  1  | gen_kl_error median < 1.3     |
|                            |      |                     |     | accuracy > 0.1                |
| colocated_reshard_async_gym | fast | colocated, reshard  |  1  | gen_kl_error median < 1.3     |
|                            |      |                     |     | accuracy > 0.1                |
| multiturn (disabled)       |      | colocated, 5 turns  |  0  | disabled: #3385 (median 1.78) |
+----------------------------+------+---------------------+-----+-------------------------------+
| L1 SingleController shard  |      |                     |     |                               |
+----------------------------+------+---------------------+-----+-------------------------------+
| grpo_megatron_generation_  | fast | non-colocated       |  0  | mult_prob_error max < 1.05    |
|   gym_single_controller    |      |                     |     | gen_kl_error median < 1.3     |
|                            |      |                     |     | reward > 0                    |
+----------------------------+------+---------------------+-----+-------------------------------+

NIGHTLIES  (nanov3 rows: name prefix grpo-nanov3-30BA3B-)
+-------+----------------------------------------------------+-------+------------------------------------+
| lane  | recipe                                             | steps | gates                              |
+-------+----------------------------------------------------+-------+------------------------------------+
| H100  | 2n8g-megatron_generation-colocated-reshard-        |   8   | gen_kl_error median < 1.3          |
|       |   async-gym (disabled)                             |       | reward > 0                         |
| H100  | 2n8g-megatron_generation-noncolocated-async-gym    |   8   | gen_kl_error median < 1.3          |
|       |                                                    |       | reward > 0                         |
|       |                                                    |       | step_time median < 1000 s          |
| GB200 | 4n4g-megatron_generation-colocated-reshard-        |   8   | gen_kl_error median < 1.3          |
|       |   async-gym (disabled)                             |       | reward > 0                         |
| GB200 | 4n4g-megatron_generation-noncolocated-async-gym    |   8   | gen_kl_error median < 1.3          |
|       |                                                    |       | reward > 0                         |
|       |                                                    |       | step_time median < 1500 s          |
| GB200 | 4n4g-megatron_generation-noncolocated-             |  10   | weight-sync + refit timings > 0    |
|       |   mxfp8-rollouts (sync; BF16 train, MXFP8          |       | gen_kl_error max < 0.15            |
|       |   rollouts via EP8 NCCL refit)                     |       | mult_prob_error median < 1.5       |
|       |                                                    |       | mult_prob_error @ step 10 < 2.0    |
|       |                                                    |       | reward > 0                         |
|       |                                                    |       | step_time median < 48 s            |
+-------+----------------------------------------------------+-------+------------------------------------+
| H100  | grpo-llama3.2-1b-instruct-1n8g-megatron_generation |  500  | mult_prob_error median < 1.1       |
|       | (colocated, lag 0; long-horizon leg)               |       | mult_prob_error @ step 500 < 1.1   |
|       |                                                    |       | reward @ step 500 > 0.1            |
|       |                                                    |       | step_time median < 12 s            |
| GB200 | grpo-llama3.2-1b-instruct-1n4g-megatron_generation |  500  | as the H100 1B leg, but            |
|       |                                                    |       | step_time median < 20 s            |
| H100  | grpo-llama3.2-1b-instruct-1n8g-                    |  500  | as the H100 1B leg (incl. its      |
|       |   megatron_generation-tq_mooncake                  |       | step_time gate), over the mooncake |
|       |                                                    |       | data plane                         |
| H100  | grpo-qwen2.5-math-1.5b-instruct-1n8g-              |  50   | mult_prob_error median < 1.1       |
|       |   megatron_generation-noncolocated-                |       | mult_prob_error @ step 50 < 1.1    |
|       |   single-controller-sync                           |       | reward > 0                         |
|       |                                                    |       | step_time median < 18 s            |
| GB200 | grpo-qwen2.5-math-1.5b-instruct-1n4g-              |  50   | as the H100 leg above, but         |
|       |   megatron_generation-noncolocated-                |       | step_time median < 30 s            |
|       |   single-controller-sync                           |       |                                    |
+-------+----------------------------------------------------+-------+------------------------------------+

MINF BUMP QUICK LEGS -- nanov3-30B-A3B, async gym, lag 1
+-------+-------------------------------------------------------------+-------+---------------------------+
| lane  | recipe (prefix grpo-nanov3-30BA3B-)                         | steps | gate                      |
+-------+-------------------------------------------------------------+-------+---------------------------+
| H100  | 2n8g-megatron_generation-noncolocated-async-gym-quick       |   3   | gen_kl_error median < 1.3 |
| H100  | 2n8g-megatron_generation-colocated-reshard-async-gym-quick  |   3   | gen_kl_error median < 1.3 |
| GB200 | 4n4g-megatron_generation-noncolocated-async-gym-quick       |   3   | gen_kl_error median < 1.3 |
| GB200 | 4n4g-megatron_generation-colocated-reshard-async-gym-quick  |   3   | gen_kl_error median < 1.3 |
+-------+-------------------------------------------------------------+-------+---------------------------+

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[copy-pr-bot]

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[tdene]

/ok to test e56ba00

[tdene]

/ok to test e34d55f

[tdene]

/ok to test d7610ff

[tdene]

/ok to test 7dcc8b2

[tdene]

/ok to test 4609054

[tdene]

/ok to test a9eb9e3

[tdene]

/ok to test cc07f72

[tdene]

/ok to test c3ac9f6

[tdene]

/ok to test a973f7b

[tdene]

/ok to test 3ca45fe

[tdene]

/ok to test 7d382aa

[tdene]

lgtm, needs a rebase since there's conflict with main.

Done

for #3570 (comment), 6 of the 10 nightly tests (except those 4 quick tests) added the step time check, I'm ok with current state, just curious are you going to add more?

I am going to add the other 4 as well, they're just having to be restarted due to spurious HF failures.

[tdene]

/ok to test 1ee606f

[tdene]

/ok to test 7e7d8e3

[cspades]

Rerunning the failed job: https://github.com/NVIDIA-NeMo/RL/actions/runs/33756537697/job/100732780253?pr=3570