Skip to content

[BugFix] Replay Buffer prefetch & SliceSampler#3322

Merged
vmoens merged 2 commits into
gh/vmoens/190/basefrom
gh/vmoens/190/head
Jan 13, 2026
Merged

[BugFix] Replay Buffer prefetch & SliceSampler#3322
vmoens merged 2 commits into
gh/vmoens/190/basefrom
gh/vmoens/190/head

Conversation

@vmoens

@vmoens vmoens commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@vmoens
Update
242c1d9 
[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3322

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Unrelated Failure

As of commit 92aecc8 with merge base 0a98e17 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 12, 2026
This was referenced Jan 12, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@github-actions

github-actions Bot commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}25$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 84.7688μs 82.5705μs 12.1109 KOps/s 12.4202 KOps/s $\color{#d91a1a}-2.49\%$
test_tensor_to_bytestream_speed[torch.save] 0.1454ms 0.1440ms 6.9430 KOps/s 7.0869 KOps/s $\color{#d91a1a}-2.03\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1386s 0.1378s 7.2555 Ops/s 7.1842 Ops/s $\color{#35bf28}+0.99\%$
test_tensor_to_bytestream_speed[numpy] 2.8387μs 2.8347μs 352.7736 KOps/s 364.5083 KOps/s $\color{#d91a1a}-3.22\%$
test_tensor_to_bytestream_speed[safetensors] 39.1318μs 38.4740μs 25.9916 KOps/s 26.1626 KOps/s $\color{#d91a1a}-0.65\%$
test_simple 0.5515s 0.5494s 1.8202 Ops/s 1.7419 Ops/s $\color{#35bf28}+4.50\%$
test_transformed 1.1276s 1.1203s 0.8926 Ops/s 0.8661 Ops/s $\color{#35bf28}+3.06\%$
test_serial 1.6683s 1.6663s 0.6001 Ops/s 0.5875 Ops/s $\color{#35bf28}+2.14\%$
test_parallel 1.2424s 1.1719s 0.8533 Ops/s 0.7834 Ops/s $\textbf{\color{#35bf28}+8.93\%}$
test_step_mdp_speed[True-True-True-True-True] 0.3321ms 44.4204μs 22.5122 KOps/s 22.4942 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-True-True-True-False] 55.7810μs 24.4790μs 40.8513 KOps/s 40.4088 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-True-False-True] 52.1510μs 24.8888μs 40.1787 KOps/s 41.0399 KOps/s $\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-True-True-False-False] 43.1710μs 13.6525μs 73.2469 KOps/s 73.7307 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[True-True-False-True-True] 83.3810μs 47.4005μs 21.0968 KOps/s 21.0424 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-False-True-False] 61.1710μs 27.2591μs 36.6851 KOps/s 36.7782 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-True-False-False-True] 57.7810μs 27.6944μs 36.1083 KOps/s 35.9163 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-True-False-False-False] 53.3610μs 16.3413μs 61.1946 KOps/s 60.6944 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-True-True-True] 0.1143ms 50.4046μs 19.8394 KOps/s 19.9159 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-False-True-True-False] 65.2710μs 29.9722μs 33.3642 KOps/s 33.1714 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-False-True-False-True] 56.5510μs 27.4131μs 36.4788 KOps/s 36.6501 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[True-False-True-False-False] 48.4710μs 16.2847μs 61.4075 KOps/s 61.0364 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-False-True-True] 81.1910μs 52.6038μs 19.0100 KOps/s 19.0189 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-False-True-False] 66.7020μs 32.6685μs 30.6105 KOps/s 30.7078 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-False-False-True] 66.9810μs 29.7287μs 33.6375 KOps/s 33.3934 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-False-False-False] 47.1310μs 18.8788μs 52.9695 KOps/s 52.7008 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-True-True-True-True] 92.7920μs 49.8278μs 20.0691 KOps/s 20.0952 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-True-True-False] 61.1110μs 29.9183μs 33.4244 KOps/s 33.0540 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-True-True-False-True] 2.8259ms 31.5049μs 31.7411 KOps/s 31.1255 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[False-True-True-False-False] 89.6820μs 18.1756μs 55.0188 KOps/s 55.3213 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-True-True] 0.1077ms 52.7139μs 18.9703 KOps/s 19.1412 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-True-False-True-False] 66.3310μs 32.8135μs 30.4753 KOps/s 30.5131 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-True-False-False-True] 68.9720μs 33.8432μs 29.5480 KOps/s 29.6833 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[False-True-False-False-False] 45.8110μs 20.6775μs 48.3617 KOps/s 48.6711 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-False-True-True-True] 0.1145ms 55.3910μs 18.0535 KOps/s 18.1470 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-True-True-False] 63.7210μs 35.8387μs 27.9028 KOps/s 27.8638 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-False-True-False-True] 59.8010μs 34.0350μs 29.3816 KOps/s 29.7934 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[False-False-True-False-False] 46.4210μs 20.6698μs 48.3798 KOps/s 48.2485 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-False-True-True] 90.9220μs 57.4394μs 17.4096 KOps/s 17.6041 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-False-False-True-False] 0.1090ms 38.2310μs 26.1568 KOps/s 26.3919 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-False-False-False-True] 71.2920μs 35.7229μs 27.9932 KOps/s 27.9806 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-False-False-False] 57.3010μs 23.1202μs 43.2523 KOps/s 43.8144 KOps/s $\color{#d91a1a}-1.28\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8814s 0.7665s 1.3047 Ops/s 1.2983 Ops/s $\color{#35bf28}+0.49\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7537s 0.6342s 1.5767 Ops/s 1.5801 Ops/s $\color{#d91a1a}-0.21\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7567s 1.6647s 0.6007 Ops/s 0.5947 Ops/s $\color{#35bf28}+1.01\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5445s 1.4575s 0.6861 Ops/s 0.6844 Ops/s $\color{#35bf28}+0.24\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0117s 1.9272s 0.5189 Ops/s 0.5196 Ops/s $\color{#d91a1a}-0.13\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.8003s 1.7160s 0.5828 Ops/s 0.5856 Ops/s $\color{#d91a1a}-0.49\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7608s 4.6702s 0.2141 Ops/s 0.2183 Ops/s $\color{#d91a1a}-1.91\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5799s 4.4566s 0.2244 Ops/s 0.2255 Ops/s $\color{#d91a1a}-0.51\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0603s 1.9612s 0.5099 Ops/s 0.5126 Ops/s $\color{#d91a1a}-0.52\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.8219s 1.7302s 0.5780 Ops/s 0.5971 Ops/s $\color{#d91a1a}-3.21\%$
test_values[generalized_advantage_estimate-True-True] 11.4817ms 11.3158ms 88.3719 Ops/s 89.3802 Ops/s $\color{#d91a1a}-1.13\%$
test_values[vec_generalized_advantage_estimate-True-True] 15.1934ms 11.3003ms 88.4932 Ops/s 85.7751 Ops/s $\color{#35bf28}+3.17\%$
test_values[td0_return_estimate-False-False] 0.2194ms 0.1320ms 7.5754 KOps/s 7.7743 KOps/s $\color{#d91a1a}-2.56\%$
test_values[td1_return_estimate-False-False] 30.6181ms 30.1749ms 33.1401 Ops/s 32.7766 Ops/s $\color{#35bf28}+1.11\%$
test_values[vec_td1_return_estimate-False-False] 12.4468ms 11.4214ms 87.5549 Ops/s 85.5565 Ops/s $\color{#35bf28}+2.34\%$
test_values[td_lambda_return_estimate-True-False] 44.9026ms 44.3141ms 22.5662 Ops/s 22.4649 Ops/s $\color{#35bf28}+0.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 11.5627ms 11.3122ms 88.4003 Ops/s 85.6959 Ops/s $\color{#35bf28}+3.16\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.3604ms 10.1962ms 98.0760 Ops/s 99.4065 Ops/s $\color{#d91a1a}-1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.8953ms 1.5300ms 653.5930 Ops/s 633.8557 Ops/s $\color{#35bf28}+3.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4942ms 0.4339ms 2.3046 KOps/s 2.2740 KOps/s $\color{#35bf28}+1.35\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.6123ms 30.0548ms 33.2726 Ops/s 31.8912 Ops/s $\color{#35bf28}+4.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1195ms 1.7336ms 576.8361 Ops/s 568.2505 Ops/s $\color{#35bf28}+1.51\%$
test_dqn_speed[False-None] 1.8024ms 1.4323ms 698.1728 Ops/s 692.9402 Ops/s $\color{#35bf28}+0.76\%$
test_dqn_speed[False-backward] 2.0258ms 1.9627ms 509.5073 Ops/s 503.9519 Ops/s $\color{#35bf28}+1.10\%$
test_dqn_speed[True-None] 0.6942ms 0.5508ms 1.8157 KOps/s 1.7857 KOps/s $\color{#35bf28}+1.68\%$
test_dqn_speed[True-backward] 1.0660ms 1.0049ms 995.1671 Ops/s 806.4491 Ops/s $\textbf{\color{#35bf28}+23.40\%}$
test_dqn_speed[reduce-overhead-None] 0.8627ms 0.5415ms 1.8469 KOps/s 1.7870 KOps/s $\color{#35bf28}+3.35\%$
test_dqn_speed[reduce-overhead-backward] 1.0347ms 0.9907ms 1.0094 KOps/s 826.5753 Ops/s $\textbf{\color{#35bf28}+22.12\%}$
test_ddpg_speed[False-None] 3.3641ms 2.9182ms 342.6720 Ops/s 330.4743 Ops/s $\color{#35bf28}+3.69\%$
test_ddpg_speed[False-backward] 4.3103ms 4.1838ms 239.0196 Ops/s 237.6212 Ops/s $\color{#35bf28}+0.59\%$
test_ddpg_speed[True-None] 1.5403ms 1.4295ms 699.5605 Ops/s 678.6580 Ops/s $\color{#35bf28}+3.08\%$
test_ddpg_speed[True-backward] 2.5363ms 2.4455ms 408.9218 Ops/s 335.5425 Ops/s $\textbf{\color{#35bf28}+21.87\%}$
test_ddpg_speed[reduce-overhead-None] 1.7908ms 1.4152ms 706.6191 Ops/s 675.5107 Ops/s $\color{#35bf28}+4.61\%$
test_ddpg_speed[reduce-overhead-backward] 2.5736ms 2.4355ms 410.5971 Ops/s 359.8545 Ops/s $\textbf{\color{#35bf28}+14.10\%}$
test_sac_speed[False-None] 8.7253ms 8.1870ms 122.1445 Ops/s 120.1619 Ops/s $\color{#35bf28}+1.65\%$
test_sac_speed[False-backward] 12.0711ms 11.5622ms 86.4888 Ops/s 86.7104 Ops/s $\color{#d91a1a}-0.26\%$
test_sac_speed[True-None] 2.5791ms 2.2079ms 452.9274 Ops/s 444.0646 Ops/s $\color{#35bf28}+2.00\%$
test_sac_speed[True-backward] 4.2830ms 4.1515ms 240.8776 Ops/s 221.3555 Ops/s $\textbf{\color{#35bf28}+8.82\%}$
test_sac_speed[reduce-overhead-None] 2.5444ms 2.1917ms 456.2675 Ops/s 447.0100 Ops/s $\color{#35bf28}+2.07\%$
test_sac_speed[reduce-overhead-backward] 4.6143ms 4.2022ms 237.9722 Ops/s 204.8931 Ops/s $\textbf{\color{#35bf28}+16.14\%}$
test_redq_speed[False-None] 11.2064ms 10.6739ms 93.6864 Ops/s 91.3603 Ops/s $\color{#35bf28}+2.55\%$
test_redq_speed[False-backward] 19.6497ms 18.5319ms 53.9610 Ops/s 52.4901 Ops/s $\color{#35bf28}+2.80\%$
test_redq_speed[True-None] 4.8564ms 4.5427ms 220.1321 Ops/s 217.7544 Ops/s $\color{#35bf28}+1.09\%$
test_redq_speed[True-backward] 10.8307ms 10.1106ms 98.9065 Ops/s 95.6506 Ops/s $\color{#35bf28}+3.40\%$
test_redq_speed[reduce-overhead-None] 5.0526ms 4.5813ms 218.2810 Ops/s 213.5011 Ops/s $\color{#35bf28}+2.24\%$
test_redq_speed[reduce-overhead-backward] 10.9808ms 10.4327ms 95.8525 Ops/s 88.7283 Ops/s $\textbf{\color{#35bf28}+8.03\%}$
test_redq_deprec_speed[False-None] 11.9738ms 11.4570ms 87.2831 Ops/s 87.5445 Ops/s $\color{#d91a1a}-0.30\%$
test_redq_deprec_speed[False-backward] 17.4986ms 16.4348ms 60.8465 Ops/s 60.4798 Ops/s $\color{#35bf28}+0.61\%$
test_redq_deprec_speed[True-None] 4.1696ms 3.8410ms 260.3502 Ops/s 264.0663 Ops/s $\color{#d91a1a}-1.41\%$
test_redq_deprec_speed[True-backward] 8.0985ms 7.8507ms 127.3772 Ops/s 126.8520 Ops/s $\color{#35bf28}+0.41\%$
test_redq_deprec_speed[reduce-overhead-None] 3.9248ms 3.7607ms 265.9097 Ops/s 269.6451 Ops/s $\color{#d91a1a}-1.39\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.2174ms 7.9458ms 125.8528 Ops/s 118.3675 Ops/s $\textbf{\color{#35bf28}+6.32\%}$
test_td3_speed[False-None] 9.4843ms 8.2768ms 120.8203 Ops/s 121.9496 Ops/s $\color{#d91a1a}-0.93\%$
test_td3_speed[False-backward] 11.6769ms 11.2275ms 89.0669 Ops/s 89.2431 Ops/s $\color{#d91a1a}-0.20\%$
test_td3_speed[True-None] 1.9469ms 1.8960ms 527.4180 Ops/s 504.5232 Ops/s $\color{#35bf28}+4.54\%$
test_td3_speed[True-backward] 3.9684ms 3.8265ms 261.3336 Ops/s 228.4485 Ops/s $\textbf{\color{#35bf28}+14.39\%}$
test_td3_speed[reduce-overhead-None] 1.9535ms 1.8856ms 530.3443 Ops/s 522.9715 Ops/s $\color{#35bf28}+1.41\%$
test_td3_speed[reduce-overhead-backward] 3.9270ms 3.8281ms 261.2281 Ops/s 234.7538 Ops/s $\textbf{\color{#35bf28}+11.28\%}$
test_cql_speed[False-None] 30.4093ms 26.9869ms 37.0550 Ops/s 36.9702 Ops/s $\color{#35bf28}+0.23\%$
test_cql_speed[False-backward] 40.4576ms 36.6766ms 27.2653 Ops/s 27.3387 Ops/s $\color{#d91a1a}-0.27\%$
test_cql_speed[True-None] 13.4395ms 12.7898ms 78.1875 Ops/s 76.2139 Ops/s $\color{#35bf28}+2.59\%$
test_cql_speed[True-backward] 21.7887ms 19.3599ms 51.6532 Ops/s 46.1772 Ops/s $\textbf{\color{#35bf28}+11.86\%}$
test_cql_speed[reduce-overhead-None] 15.7406ms 12.9148ms 77.4304 Ops/s 74.7925 Ops/s $\color{#35bf28}+3.53\%$
test_cql_speed[reduce-overhead-backward] 19.6142ms 19.1138ms 52.3183 Ops/s 50.5345 Ops/s $\color{#35bf28}+3.53\%$
test_a2c_speed[False-None] 5.8011ms 5.5786ms 179.2569 Ops/s 173.4114 Ops/s $\color{#35bf28}+3.37\%$
test_a2c_speed[False-backward] 12.6363ms 12.2734ms 81.4769 Ops/s 78.8233 Ops/s $\color{#35bf28}+3.37\%$
test_a2c_speed[True-None] 3.9587ms 3.7772ms 264.7487 Ops/s 248.4522 Ops/s $\textbf{\color{#35bf28}+6.56\%}$
test_a2c_speed[True-backward] 9.1927ms 8.8934ms 112.4429 Ops/s 105.9543 Ops/s $\textbf{\color{#35bf28}+6.12\%}$
test_a2c_speed[reduce-overhead-None] 4.0189ms 3.8546ms 259.4330 Ops/s 259.5089 Ops/s $\color{#d91a1a}-0.03\%$
test_a2c_speed[reduce-overhead-backward] 9.4419ms 9.0632ms 110.3357 Ops/s 105.1403 Ops/s $\color{#35bf28}+4.94\%$
test_ppo_speed[False-None] 6.8715ms 6.0932ms 164.1177 Ops/s 160.7063 Ops/s $\color{#35bf28}+2.12\%$
test_ppo_speed[False-backward] 13.4162ms 12.9712ms 77.0938 Ops/s 75.4534 Ops/s $\color{#35bf28}+2.17\%$
test_ppo_speed[True-None] 3.8461ms 3.7213ms 268.7261 Ops/s 259.4078 Ops/s $\color{#35bf28}+3.59\%$
test_ppo_speed[True-backward] 8.8962ms 8.6569ms 115.5149 Ops/s 112.7234 Ops/s $\color{#35bf28}+2.48\%$
test_ppo_speed[reduce-overhead-None] 3.8463ms 3.7244ms 268.4976 Ops/s 268.0826 Ops/s $\color{#35bf28}+0.15\%$
test_ppo_speed[reduce-overhead-backward] 9.2647ms 8.9819ms 111.3354 Ops/s 104.0285 Ops/s $\textbf{\color{#35bf28}+7.02\%}$
test_reinforce_speed[False-None] 5.0146ms 4.7591ms 210.1221 Ops/s 208.6945 Ops/s $\color{#35bf28}+0.68\%$
test_reinforce_speed[False-backward] 7.8391ms 7.6937ms 129.9769 Ops/s 128.8934 Ops/s $\color{#35bf28}+0.84\%$
test_reinforce_speed[True-None] 3.1661ms 2.9537ms 338.5623 Ops/s 326.0159 Ops/s $\color{#35bf28}+3.85\%$
test_reinforce_speed[True-backward] 8.2489ms 7.9336ms 126.0454 Ops/s 117.0968 Ops/s $\textbf{\color{#35bf28}+7.64\%}$
test_reinforce_speed[reduce-overhead-None] 3.1331ms 2.9476ms 339.2578 Ops/s 325.5374 Ops/s $\color{#35bf28}+4.21\%$
test_reinforce_speed[reduce-overhead-backward] 8.3107ms 8.1237ms 123.0971 Ops/s 119.8647 Ops/s $\color{#35bf28}+2.70\%$
test_iql_speed[False-None] 25.5002ms 20.7669ms 48.1535 Ops/s 46.9522 Ops/s $\color{#35bf28}+2.56\%$
test_iql_speed[False-backward] 32.3460ms 31.5114ms 31.7346 Ops/s 31.4435 Ops/s $\color{#35bf28}+0.93\%$
test_iql_speed[True-None] 9.0139ms 8.7618ms 114.1324 Ops/s 111.0911 Ops/s $\color{#35bf28}+2.74\%$
test_iql_speed[True-backward] 18.0995ms 17.1786ms 58.2119 Ops/s 56.6776 Ops/s $\color{#35bf28}+2.71\%$
test_iql_speed[reduce-overhead-None] 9.3718ms 8.8287ms 113.2667 Ops/s 110.4941 Ops/s $\color{#35bf28}+2.51\%$
test_iql_speed[reduce-overhead-backward] 18.0894ms 17.5746ms 56.9004 Ops/s 54.9584 Ops/s $\color{#35bf28}+3.53\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.4877ms 6.1837ms 161.7142 Ops/s 162.1842 Ops/s $\color{#d91a1a}-0.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5923ms 0.2968ms 3.3698 KOps/s 3.3766 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6933ms 0.2730ms 3.6631 KOps/s 3.6022 KOps/s $\color{#35bf28}+1.69\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0317ms 5.8050ms 172.2646 Ops/s 169.8071 Ops/s $\color{#35bf28}+1.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7538ms 0.3029ms 3.3012 KOps/s 2.9589 KOps/s $\textbf{\color{#35bf28}+11.57\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5593ms 0.2712ms 3.6867 KOps/s 3.2916 KOps/s $\textbf{\color{#35bf28}+12.00\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6359ms 1.3554ms 737.7935 Ops/s 667.5913 Ops/s $\textbf{\color{#35bf28}+10.52\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4685ms 1.2572ms 795.4446 Ops/s 713.9236 Ops/s $\textbf{\color{#35bf28}+11.42\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 12.6542ms 6.0871ms 164.2826 Ops/s 167.8526 Ops/s $\color{#d91a1a}-2.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9621ms 0.4828ms 2.0711 KOps/s 2.0584 KOps/s $\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7416ms 0.4344ms 2.3019 KOps/s 2.0387 KOps/s $\textbf{\color{#35bf28}+12.91\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1042ms 5.8087ms 172.1567 Ops/s 172.6359 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.9182ms 0.3208ms 3.1168 KOps/s 2.8594 KOps/s $\textbf{\color{#35bf28}+9.00\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5546ms 0.3167ms 3.1578 KOps/s 3.5980 KOps/s $\textbf{\color{#d91a1a}-12.23\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9816ms 5.7676ms 173.3809 Ops/s 175.7829 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0672ms 0.2969ms 3.3686 KOps/s 3.0780 KOps/s $\textbf{\color{#35bf28}+9.44\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4857ms 0.2738ms 3.6526 KOps/s 3.4012 KOps/s $\textbf{\color{#35bf28}+7.39\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1152ms 5.9472ms 168.1476 Ops/s 167.7457 Ops/s $\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9239ms 0.4927ms 2.0297 KOps/s 2.1446 KOps/s $\textbf{\color{#d91a1a}-5.36\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.8108ms 0.4549ms 2.1982 KOps/s 2.3013 KOps/s $\color{#d91a1a}-4.48\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.5029ms 5.0267ms 198.9379 Ops/s 195.1313 Ops/s $\color{#35bf28}+1.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.5083ms 2.3289ms 429.3961 Ops/s 466.6572 Ops/s $\textbf{\color{#d91a1a}-7.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.8051ms 1.2603ms 793.4547 Ops/s 856.2060 Ops/s $\textbf{\color{#d91a1a}-7.33\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.6380s 17.7723ms 56.2673 Ops/s 193.5021 Ops/s $\textbf{\color{#d91a1a}-70.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.3125ms 2.0643ms 484.4142 Ops/s 484.8281 Ops/s $\color{#d91a1a}-0.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.0445ms 1.2218ms 818.4384 Ops/s 799.7180 Ops/s $\color{#35bf28}+2.34\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.1163ms 5.2563ms 190.2461 Ops/s 49.8478 Ops/s $\textbf{\color{#35bf28}+281.65\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.9877ms 2.2682ms 440.8731 Ops/s 447.1442 Ops/s $\color{#d91a1a}-1.40\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6829ms 1.4180ms 705.2077 Ops/s 716.9590 Ops/s $\color{#d91a1a}-1.64\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.3768ms 34.8066ms 28.7302 Ops/s 29.0604 Ops/s $\color{#d91a1a}-1.14\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 23.3944ms 18.5705ms 53.8487 Ops/s 55.6250 Ops/s $\color{#d91a1a}-3.19\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 37.8344ms 35.5915ms 28.0966 Ops/s 27.8573 Ops/s $\color{#35bf28}+0.86\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.2223ms 18.6203ms 53.7047 Ops/s 54.2935 Ops/s $\color{#d91a1a}-1.08\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 39.2052ms 37.5349ms 26.6418 Ops/s 26.5308 Ops/s $\color{#35bf28}+0.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.5244ms 19.9111ms 50.2231 Ops/s 49.9435 Ops/s $\color{#35bf28}+0.56\%$

@vmoens
Update
92aecc8 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jan 13, 2026
@vmoens
[BugFix] Replay Buffer prefetch & SliceSampler
9d34dbe 
Fix pickling for `ReplayBuffer` with prefetch by removing non-picklable
prefetch objects and recreating them on unpickle.
Fix logging typo in `SliceSampler`.


ghstack-source-id: 7a8b94a
Pull-Request: #3322
@vmoens vmoens merged commit 92aecc8 into gh/vmoens/190/base Jan 13, 2026
101 of 106 checks passed
@vmoens vmoens deleted the gh/vmoens/190/head branch January 13, 2026 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens