Skip to content

[Benchmark] Convert plain scripts to pytest-discoverable test files - #1594

Closed
vmoens wants to merge 3 commits into
gh/vmoens/77/basefrom
gh/vmoens/77/head
Closed

[Benchmark] Convert plain scripts to pytest-discoverable test files#1594
vmoens wants to merge 3 commits into
gh/vmoens/77/basefrom
gh/vmoens/77/head

Conversation

@vmoens

@vmoens vmoens commented Feb 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Stack from ghstack (oldest at bottom):

Rename and restructure all benchmark scripts in benchmarks/ so they are
discoverable by pytest. Each file now uses pytest.mark.skipif to skip
gracefully when infrastructure (NCCL nodes, Redis, ImageNet, etc.) is
unavailable.

  • bench_distributed.py -> test_distributed_benchmarks.py
  • fx_benchmarks.py -> test_fx_benchmarks.py
  • storage/bench_redis.py -> storage/test_redis_benchmarks.py
  • distributed/dataloading.py -> distributed/test_dataloading.py

Co-authored-by: Cursor cursoragent@cursor.com

@vmoens
Update
3e51eee 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Feb 19, 2026
@vmoens @cursoragent
[Benchmark] Convert plain scripts to pytest-discoverable test files
e0ed9a3 
Rename and restructure all benchmark scripts in benchmarks/ so they are
discoverable by pytest.  Each file now uses pytest.mark.skipif to skip
gracefully when infrastructure (NCCL nodes, Redis, ImageNet, etc.) is
unavailable.

- bench_distributed.py -> test_distributed_benchmarks.py
- fx_benchmarks.py -> test_fx_benchmarks.py
- storage/bench_redis.py -> storage/test_redis_benchmarks.py
- distributed/dataloading.py -> distributed/test_dataloading.py

Co-authored-by: Cursor <cursoragent@cursor.com>
ghstack-source-id: 86cd248
Pull-Request: #1594
@vmoens vmoens mentioned this pull request Feb 19, 2026
Merged
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 19, 2026
This was referenced Feb 19, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@github-actions

github-actions Bot commented Feb 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 243. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 40.0310μs 14.6314μs 68.3464 KOps/s 68.1919 KOps/s $\color{#35bf28}+0.23\%$
test_plain_set_stack_nested 39.9810μs 15.0631μs 66.3874 KOps/s 66.7296 KOps/s $\color{#d91a1a}-0.51\%$
test_plain_set_nested_inplace 45.6110μs 16.4054μs 60.9557 KOps/s 60.0672 KOps/s $\color{#35bf28}+1.48\%$
test_plain_set_stack_nested_inplace 44.6110μs 16.2555μs 61.5177 KOps/s 60.8167 KOps/s $\color{#35bf28}+1.15\%$
test_items 28.4810μs 5.8698μs 170.3638 KOps/s 172.9114 KOps/s $\color{#d91a1a}-1.47\%$
test_items_nested 0.5116ms 0.4519ms 2.2131 KOps/s 2.1827 KOps/s $\color{#35bf28}+1.39\%$
test_items_nested_locked 0.5271ms 0.4558ms 2.1939 KOps/s 2.1482 KOps/s $\color{#35bf28}+2.13\%$
test_items_nested_leaf 0.1283ms 96.3688μs 10.3768 KOps/s 10.2915 KOps/s $\color{#35bf28}+0.83\%$
test_items_stack_nested 0.5170ms 0.4545ms 2.2000 KOps/s 2.1559 KOps/s $\color{#35bf28}+2.04\%$
test_items_stack_nested_leaf 0.1259ms 98.2499μs 10.1781 KOps/s 10.3457 KOps/s $\color{#d91a1a}-1.62\%$
test_items_stack_nested_locked 0.5137ms 0.4595ms 2.1761 KOps/s 2.1492 KOps/s $\color{#35bf28}+1.25\%$
test_keys 37.5600μs 4.1710μs 239.7512 KOps/s 232.5592 KOps/s $\color{#35bf28}+3.09\%$
test_keys_nested 0.1568ms 0.1193ms 8.3818 KOps/s 8.2418 KOps/s $\color{#35bf28}+1.70\%$
test_keys_nested_locked 0.7263ms 0.1290ms 7.7512 KOps/s 7.7683 KOps/s $\color{#d91a1a}-0.22\%$
test_keys_nested_leaf 0.1424ms 0.1095ms 9.1292 KOps/s 8.9520 KOps/s $\color{#35bf28}+1.98\%$
test_keys_stack_nested 0.1683ms 0.1200ms 8.3332 KOps/s 8.2658 KOps/s $\color{#35bf28}+0.81\%$
test_keys_stack_nested_leaf 0.1425ms 0.1106ms 9.0390 KOps/s 8.9898 KOps/s $\color{#35bf28}+0.55\%$
test_keys_stack_nested_locked 0.1846ms 0.1282ms 7.7979 KOps/s 7.7122 KOps/s $\color{#35bf28}+1.11\%$
test_values 8.5760μs 1.0136μs 986.6108 KOps/s 975.8276 KOps/s $\color{#35bf28}+1.11\%$
test_values_nested 66.6310μs 47.9665μs 20.8479 KOps/s 20.6542 KOps/s $\color{#35bf28}+0.94\%$
test_values_nested_locked 82.7710μs 51.1638μs 19.5451 KOps/s 19.5393 KOps/s $\color{#35bf28}+0.03\%$
test_values_nested_leaf 89.2110μs 54.9893μs 18.1854 KOps/s 18.2829 KOps/s $\color{#d91a1a}-0.53\%$
test_values_stack_nested 74.0820μs 47.9224μs 20.8671 KOps/s 20.7596 KOps/s $\color{#35bf28}+0.52\%$
test_values_stack_nested_leaf 92.7010μs 54.9399μs 18.2017 KOps/s 18.3002 KOps/s $\color{#d91a1a}-0.54\%$
test_values_stack_nested_locked 85.6320μs 50.7206μs 19.7159 KOps/s 19.5583 KOps/s $\color{#35bf28}+0.81\%$
test_membership 4.6367μs 0.8478μs 1.1796 MOps/s 1.1713 MOps/s $\color{#35bf28}+0.70\%$
test_membership_nested 42.0810μs 2.8897μs 346.0530 KOps/s 338.7165 KOps/s $\color{#35bf28}+2.17\%$
test_membership_nested_leaf 26.5900μs 2.8843μs 346.7029 KOps/s 337.6755 KOps/s $\color{#35bf28}+2.67\%$
test_membership_stacked_nested 39.0610μs 2.9556μs 338.3421 KOps/s 338.6652 KOps/s $\color{#d91a1a}-0.10\%$
test_membership_stacked_nested_leaf 25.8010μs 2.9459μs 339.4595 KOps/s 337.3698 KOps/s $\color{#35bf28}+0.62\%$
test_membership_nested_last 37.6500μs 4.3500μs 229.8828 KOps/s 226.9130 KOps/s $\color{#35bf28}+1.31\%$
test_membership_nested_leaf_last 27.0700μs 4.3610μs 229.3047 KOps/s 225.2548 KOps/s $\color{#35bf28}+1.80\%$
test_membership_stacked_nested_last 47.0610μs 4.3200μs 231.4826 KOps/s 229.0667 KOps/s $\color{#35bf28}+1.05\%$
test_membership_stacked_nested_leaf_last 32.2610μs 4.3243μs 231.2535 KOps/s 230.2482 KOps/s $\color{#35bf28}+0.44\%$
test_nested_getleaf 50.9910μs 21.2388μs 47.0836 KOps/s 45.2365 KOps/s $\color{#35bf28}+4.08\%$
test_nested_get 47.7810μs 20.2329μs 49.4243 KOps/s 48.4480 KOps/s $\color{#35bf28}+2.02\%$
test_stacked_getleaf 47.6910μs 21.0888μs 47.4185 KOps/s 46.0304 KOps/s $\color{#35bf28}+3.02\%$
test_stacked_get 54.0610μs 20.3888μs 49.0465 KOps/s 48.9321 KOps/s $\color{#35bf28}+0.23\%$
test_nested_getitemleaf 84.3420μs 21.6742μs 46.1378 KOps/s 44.4203 KOps/s $\color{#35bf28}+3.87\%$
test_nested_getitem 54.1410μs 20.5975μs 48.5495 KOps/s 47.4213 KOps/s $\color{#35bf28}+2.38\%$
test_stacked_getitemleaf 54.9310μs 21.7435μs 45.9907 KOps/s 44.5571 KOps/s $\color{#35bf28}+3.22\%$
test_stacked_getitem 46.2210μs 20.8597μs 47.9393 KOps/s 46.7192 KOps/s $\color{#35bf28}+2.61\%$
test_lock_nested 0.5519ms 0.4740ms 2.1096 KOps/s 2.0853 KOps/s $\color{#35bf28}+1.17\%$
test_lock_stack_nested 0.5216ms 0.4792ms 2.0868 KOps/s 2.0699 KOps/s $\color{#35bf28}+0.82\%$
test_unlock_nested 0.4544ms 0.3839ms 2.6051 KOps/s 2.6122 KOps/s $\color{#d91a1a}-0.27\%$
test_unlock_stack_nested 0.4577ms 0.3851ms 2.5969 KOps/s 2.5717 KOps/s $\color{#35bf28}+0.98\%$
test_flatten_speed 0.1843ms 0.1226ms 8.1566 KOps/s 8.0814 KOps/s $\color{#35bf28}+0.93\%$
test_unflatten_speed 0.6319ms 0.5721ms 1.7481 KOps/s 1.7258 KOps/s $\color{#35bf28}+1.29\%$
test_common_ops 0.8699ms 0.6821ms 1.4661 KOps/s 1.4454 KOps/s $\color{#35bf28}+1.44\%$
test_creation 0.1479ms 2.9174μs 342.7729 KOps/s 341.9721 KOps/s $\color{#35bf28}+0.23\%$
test_creation_empty 22.3710μs 6.1591μs 162.3605 KOps/s 161.7365 KOps/s $\color{#35bf28}+0.39\%$
test_creation_nested_1 38.6810μs 10.4826μs 95.3962 KOps/s 94.2970 KOps/s $\color{#35bf28}+1.17\%$
test_creation_nested_2 48.3010μs 11.8477μs 84.4043 KOps/s 83.2814 KOps/s $\color{#35bf28}+1.35\%$
test_creation_many_keys[10] 60.2910μs 18.3815μs 54.4026 KOps/s 54.0552 KOps/s $\color{#35bf28}+0.64\%$
test_creation_many_keys[50] 0.1179ms 78.2095μs 12.7862 KOps/s 12.7206 KOps/s $\color{#35bf28}+0.52\%$
test_creation_many_keys[100] 0.1912ms 0.1538ms 6.5013 KOps/s 6.5032 KOps/s $\color{#d91a1a}-0.03\%$
test_creation_nested_many_keys[10] 63.6220μs 39.5613μs 25.2772 KOps/s 25.2683 KOps/s $\color{#35bf28}+0.04\%$
test_creation_nested_many_keys[50] 0.2527ms 0.1605ms 6.2323 KOps/s 6.1791 KOps/s $\color{#35bf28}+0.86\%$
test_clone 52.9310μs 13.4198μs 74.5167 KOps/s 73.7717 KOps/s $\color{#35bf28}+1.01\%$
test_getitem[int] 1.8066ms 14.5364μs 68.7929 KOps/s 55.4222 KOps/s $\textbf{\color{#35bf28}+24.13\%}$
test_getitem[slice_int] 0.1434ms 24.5413μs 40.7476 KOps/s 40.3514 KOps/s $\color{#35bf28}+0.98\%$
test_getitem[range] 0.1743ms 62.4201μs 16.0205 KOps/s 16.0159 KOps/s $\color{#35bf28}+0.03\%$
test_getitem[tuple] 0.1464ms 23.7519μs 42.1019 KOps/s 41.7595 KOps/s $\color{#35bf28}+0.82\%$
test_getitem[list] 0.1857ms 57.2909μs 17.4548 KOps/s 17.2413 KOps/s $\color{#35bf28}+1.24\%$
test_setitem_dim[int] 48.2610μs 27.0928μs 36.9101 KOps/s 37.4763 KOps/s $\color{#d91a1a}-1.51\%$
test_setitem_dim[slice_int] 69.0710μs 47.0934μs 21.2344 KOps/s 22.3949 KOps/s $\textbf{\color{#d91a1a}-5.18\%}$
test_setitem_dim[range] 0.1250ms 97.0609μs 10.3028 KOps/s 10.4812 KOps/s $\color{#d91a1a}-1.70\%$
test_setitem_dim[tuple] 74.9210μs 42.7687μs 23.3816 KOps/s 23.9962 KOps/s $\color{#d91a1a}-2.56\%$
test_setitem 56.2310μs 17.6166μs 56.7646 KOps/s 55.0892 KOps/s $\color{#35bf28}+3.04\%$
test_set 53.5510μs 17.0491μs 58.6542 KOps/s 57.8831 KOps/s $\color{#35bf28}+1.33\%$
test_set_shared 0.5568ms 0.2135ms 4.6833 KOps/s 4.8013 KOps/s $\color{#d91a1a}-2.46\%$
test_update 0.2127ms 21.8406μs 45.7864 KOps/s 46.3381 KOps/s $\color{#d91a1a}-1.19\%$
test_update_nested 75.7310μs 33.7743μs 29.6083 KOps/s 29.5703 KOps/s $\color{#35bf28}+0.13\%$
test_update__nested 0.4997ms 34.6183μs 28.8865 KOps/s 28.5664 KOps/s $\color{#35bf28}+1.12\%$
test_set_nested 51.6110μs 18.8297μs 53.1076 KOps/s 51.9807 KOps/s $\color{#35bf28}+2.17\%$
test_set_nested_new 58.2120μs 23.8704μs 41.8930 KOps/s 40.7384 KOps/s $\color{#35bf28}+2.83\%$
test_select 80.9210μs 42.1752μs 23.7106 KOps/s 23.6032 KOps/s $\color{#35bf28}+0.46\%$
test_select_nested 0.1054ms 73.6164μs 13.5839 KOps/s 13.2794 KOps/s $\color{#35bf28}+2.29\%$
test_exclude_nested 0.1218ms 91.3510μs 10.9468 KOps/s 10.7697 KOps/s $\color{#35bf28}+1.64\%$
test_empty[True] 0.4328ms 0.3846ms 2.5998 KOps/s 2.5281 KOps/s $\color{#35bf28}+2.83\%$
test_empty[False] 9.6175μs 1.3146μs 760.6906 KOps/s 750.3731 KOps/s $\color{#35bf28}+1.37\%$
test_to 0.1026ms 71.0303μs 14.0785 KOps/s 13.6414 KOps/s $\color{#35bf28}+3.20\%$
test_to_nonblocking 0.1115ms 64.0024μs 15.6244 KOps/s 15.4966 KOps/s $\color{#35bf28}+0.82\%$
test_unbind_speed 0.3622ms 0.3269ms 3.0592 KOps/s 3.0536 KOps/s $\color{#35bf28}+0.18\%$
test_unbind_speed_stack0 0.3718ms 0.3259ms 3.0685 KOps/s 3.0524 KOps/s $\color{#35bf28}+0.53\%$
test_unbind_speed_stack1 0.1042s 0.8351ms 1.1974 KOps/s 1.1830 KOps/s $\color{#35bf28}+1.21\%$
test_split 0.1041s 1.2652ms 790.3721 Ops/s 786.8926 Ops/s $\color{#35bf28}+0.44\%$
test_chunk 0.1044s 1.2055ms 829.5481 Ops/s 924.2781 Ops/s $\textbf{\color{#d91a1a}-10.25\%}$
test_to_cpu_blocking 28.5433ms 28.3856ms 35.2291 Ops/s 35.1426 Ops/s $\color{#35bf28}+0.25\%$
test_to_cpu_global_sync 11.3110ms 11.2006ms 89.2812 Ops/s 80.7640 Ops/s $\textbf{\color{#35bf28}+10.55\%}$
test_to_cpu_event_sync 12.3538ms 12.1663ms 82.1945 Ops/s 82.2154 Ops/s $\color{#d91a1a}-0.03\%$
test_to_cpu_default 0.1162s 13.4268ms 74.4778 Ops/s 82.0822 Ops/s $\textbf{\color{#d91a1a}-9.26\%}$
test_consolidate[False-None] 4.1786ms 4.0952ms 244.1877 Ops/s 220.4490 Ops/s $\textbf{\color{#35bf28}+10.77\%}$
test_consolidate[default-None] 2.4243ms 2.0155ms 496.1513 Ops/s 484.4833 Ops/s $\color{#35bf28}+2.41\%$
test_consolidate[reduce-overhead-None] 2.3503ms 1.9235ms 519.8984 Ops/s 502.2553 Ops/s $\color{#35bf28}+3.51\%$
test_consolidate_njt[False-None] 8.8632ms 8.4101ms 118.9042 Ops/s 119.4046 Ops/s $\color{#d91a1a}-0.42\%$
test_to[False-False-None] 2.4885ms 2.0623ms 484.8947 Ops/s 481.9368 Ops/s $\color{#35bf28}+0.61\%$
test_to[True-False-None] 2.4103ms 1.9455ms 514.0025 Ops/s 516.8778 Ops/s $\color{#d91a1a}-0.56\%$
test_to[within-False-None] 6.5141ms 6.0973ms 164.0058 Ops/s 163.7726 Ops/s $\color{#35bf28}+0.14\%$
test_to[True-default-None] 8.1092ms 7.5845ms 131.8486 Ops/s 124.6287 Ops/s $\textbf{\color{#35bf28}+5.79\%}$
test_to_njt[False-False-None] 8.9418ms 8.5566ms 116.8685 Ops/s 112.2114 Ops/s $\color{#35bf28}+4.15\%$
test_to_njt[True-False-None] 7.4132ms 6.9519ms 143.8454 Ops/s 138.7298 Ops/s $\color{#35bf28}+3.69\%$
test_to_njt[within-False-None] 15.9263ms 15.4440ms 64.7502 Ops/s 63.0070 Ops/s $\color{#35bf28}+2.77\%$
test_creation[device0] 0.2691ms 0.1157ms 8.6446 KOps/s 8.1316 KOps/s $\textbf{\color{#35bf28}+6.31\%}$
test_creation_from_tensor 0.5471ms 0.1129ms 8.8554 KOps/s 8.3648 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_add_one[memmap_tensor0] 0.1642ms 6.8463μs 146.0638 KOps/s 144.0708 KOps/s $\color{#35bf28}+1.38\%$
test_contiguous[memmap_tensor0] 12.6400μs 0.6714μs 1.4895 MOps/s 1.9562 MOps/s $\textbf{\color{#d91a1a}-23.85\%}$
test_stack[memmap_tensor0] 35.6810μs 4.7129μs 212.1841 KOps/s 218.0637 KOps/s $\color{#d91a1a}-2.70\%$
test_memmaptd_index 1.0219ms 0.2627ms 3.8064 KOps/s 3.7803 KOps/s $\color{#35bf28}+0.69\%$
test_memmaptd_index_astensor 0.7908ms 0.3603ms 2.7753 KOps/s 2.7756 KOps/s $\color{#d91a1a}-0.01\%$
test_memmaptd_index_op 1.0388ms 0.6138ms 1.6292 KOps/s 1.6287 KOps/s $\color{#35bf28}+0.03\%$
test_serialize_model 0.3151s 0.1630s 6.1332 Ops/s 7.3016 Ops/s $\textbf{\color{#d91a1a}-16.00\%}$
test_serialize_model_pickle 1.3498s 1.1920s 0.8389 Ops/s 0.8238 Ops/s $\color{#35bf28}+1.84\%$
test_serialize_weights 0.1407s 0.1367s 7.3163 Ops/s 7.2655 Ops/s $\color{#35bf28}+0.70\%$
test_serialize_weights_returnearly 0.4429s 94.2847ms 10.6062 Ops/s 5.7122 Ops/s $\textbf{\color{#35bf28}+85.68\%}$
test_serialize_weights_pickle 1.3659s 1.1964s 0.8359 Ops/s 0.8233 Ops/s $\color{#35bf28}+1.52\%$
test_reshape_pytree 0.2078ms 33.3373μs 29.9964 KOps/s 29.9865 KOps/s $\color{#35bf28}+0.03\%$
test_reshape_td 73.0210μs 44.4606μs 22.4918 KOps/s 22.6674 KOps/s $\color{#d91a1a}-0.77\%$
test_view_pytree 0.2200ms 32.9584μs 30.3412 KOps/s 30.5674 KOps/s $\color{#d91a1a}-0.74\%$
test_view_td 79.7310μs 52.3221μs 19.1124 KOps/s 19.2472 KOps/s $\color{#d91a1a}-0.70\%$
test_unbind_pytree 0.2313ms 36.8395μs 27.1448 KOps/s 26.9242 KOps/s $\color{#35bf28}+0.82\%$
test_unbind_td 0.1976ms 49.6290μs 20.1495 KOps/s 20.4156 KOps/s $\color{#d91a1a}-1.30\%$
test_split_pytree 0.2004ms 42.4864μs 23.5369 KOps/s 23.3838 KOps/s $\color{#35bf28}+0.66\%$
test_split_td 0.1348ms 65.3515μs 15.3019 KOps/s 15.3465 KOps/s $\color{#d91a1a}-0.29\%$
test_add_pytree 0.2418ms 42.6774μs 23.4316 KOps/s 23.1065 KOps/s $\color{#35bf28}+1.41\%$
test_add_td 99.9110μs 54.1157μs 18.4789 KOps/s 17.8705 KOps/s $\color{#35bf28}+3.40\%$
test_compile_add_one_nested[tensordict-compile] 0.1913ms 0.1408ms 7.1020 KOps/s 6.8254 KOps/s $\color{#35bf28}+4.05\%$
test_compile_add_one_nested[tensordict-eager] 0.3034ms 0.1929ms 5.1846 KOps/s 5.2612 KOps/s $\color{#d91a1a}-1.45\%$
test_compile_add_one_nested[pytree-compile] 0.2897ms 0.1086ms 9.2117 KOps/s 9.0883 KOps/s $\color{#35bf28}+1.36\%$
test_compile_add_one_nested[pytree-eager] 0.6036ms 0.1823ms 5.4851 KOps/s 5.4429 KOps/s $\color{#35bf28}+0.78\%$
test_compile_copy_nested[tensordict-compile] 0.4588ms 30.4905μs 32.7971 KOps/s 31.9273 KOps/s $\color{#35bf28}+2.72\%$
test_compile_copy_nested[tensordict-eager] 0.4733ms 51.9584μs 19.2462 KOps/s 18.9742 KOps/s $\color{#35bf28}+1.43\%$
test_compile_copy_nested[pytree-compile] 0.1332ms 9.6551μs 103.5720 KOps/s 101.1571 KOps/s $\color{#35bf28}+2.39\%$
test_compile_copy_nested[pytree-eager] 0.5077ms 69.4004μs 14.4091 KOps/s 14.3219 KOps/s $\color{#35bf28}+0.61\%$
test_compile_add_one_flat[tensordict-compile] 0.2738ms 0.1766ms 5.6620 KOps/s 5.4324 KOps/s $\color{#35bf28}+4.23\%$
test_compile_add_one_flat[tensordict-eager] 0.7140ms 0.2546ms 3.9278 KOps/s 3.9023 KOps/s $\color{#35bf28}+0.65\%$
test_compile_add_one_flat[tensorclass-compile] 0.3125ms 0.1159ms 8.6260 KOps/s 8.3716 KOps/s $\color{#35bf28}+3.04\%$
test_compile_add_one_flat[tensorclass-eager] 0.5014ms 69.3577μs 14.4180 KOps/s 14.4475 KOps/s $\color{#d91a1a}-0.20\%$
test_compile_add_one_flat[pytree-compile] 0.2253ms 0.1583ms 6.3161 KOps/s 6.1430 KOps/s $\color{#35bf28}+2.82\%$
test_compile_add_one_flat[pytree-eager] 0.9758ms 0.5371ms 1.8617 KOps/s 1.8525 KOps/s $\color{#35bf28}+0.50\%$
test_compile_add_self_flat[tensordict-eager] 0.7336ms 0.3083ms 3.2440 KOps/s 3.1663 KOps/s $\color{#35bf28}+2.46\%$
test_compile_add_self_flat[tensordict-compile] 0.6304ms 0.1804ms 5.5444 KOps/s 5.1547 KOps/s $\textbf{\color{#35bf28}+7.56\%}$
test_compile_add_self_flat[tensorclass-eager] 0.1877ms 84.8154μs 11.7903 KOps/s 11.8177 KOps/s $\color{#d91a1a}-0.23\%$
test_compile_add_self_flat[tensorclass-compile] 0.2700ms 0.1252ms 7.9847 KOps/s 7.9931 KOps/s $\color{#d91a1a}-0.10\%$
test_compile_add_self_flat[pytree-eager] 0.6692ms 0.4445ms 2.2500 KOps/s 2.2541 KOps/s $\color{#d91a1a}-0.18\%$
test_compile_add_self_flat[pytree-compile] 0.2423ms 0.1648ms 6.0692 KOps/s 6.1647 KOps/s $\color{#d91a1a}-1.55\%$
test_compile_copy_flat[tensordict-compile] 0.1225ms 25.0045μs 39.9927 KOps/s 39.5475 KOps/s $\color{#35bf28}+1.13\%$
test_compile_copy_flat[tensordict-eager] 0.1321ms 40.4775μs 24.7051 KOps/s 24.1942 KOps/s $\color{#35bf28}+2.11\%$
test_compile_copy_flat[pytree-compile] 0.4388ms 10.8285μs 92.3489 KOps/s 93.4850 KOps/s $\color{#d91a1a}-1.22\%$
test_compile_copy_flat[pytree-eager] 0.4708ms 52.2312μs 19.1456 KOps/s 19.0937 KOps/s $\color{#35bf28}+0.27\%$
test_compile_assign_and_add[tensordict-compile] 2.0053ms 0.1737ms 5.7587 KOps/s 5.4960 KOps/s $\color{#35bf28}+4.78\%$
test_compile_assign_and_add[tensordict-eager] 3.4646ms 3.2856ms 304.3556 Ops/s 306.1054 Ops/s $\color{#d91a1a}-0.57\%$
test_compile_assign_and_add[pytree-compile] 1.9691ms 0.1620ms 6.1716 KOps/s 6.0505 KOps/s $\color{#35bf28}+2.00\%$
test_compile_assign_and_add[pytree-eager] 2.9602ms 2.8227ms 354.2740 Ops/s 355.8823 Ops/s $\color{#d91a1a}-0.45\%$
test_compile_indexing[tensor-tensordict-compile] 0.2223ms 0.1087ms 9.2004 KOps/s 8.8942 KOps/s $\color{#35bf28}+3.44\%$
test_compile_indexing[tensor-tensordict-eager] 0.3102ms 73.2298μs 13.6556 KOps/s 13.6814 KOps/s $\color{#d91a1a}-0.19\%$
test_compile_indexing[tensor-tensorclass-compile] 0.1639ms 95.7527μs 10.4436 KOps/s 10.2074 KOps/s $\color{#35bf28}+2.31\%$
test_compile_indexing[tensor-tensorclass-eager] 0.2568ms 45.0692μs 22.1881 KOps/s 21.6521 KOps/s $\color{#35bf28}+2.48\%$
test_compile_indexing[tensor-pytree-compile] 0.1988ms 96.3414μs 10.3798 KOps/s 10.1433 KOps/s $\color{#35bf28}+2.33\%$
test_compile_indexing[tensor-pytree-eager] 0.2867ms 44.9094μs 22.2671 KOps/s 21.5235 KOps/s $\color{#35bf28}+3.45\%$
test_compile_indexing[slice-tensordict-compile] 0.2029ms 56.7496μs 17.6213 KOps/s 17.4504 KOps/s $\color{#35bf28}+0.98\%$
test_compile_indexing[slice-tensordict-eager] 0.2261ms 28.1903μs 35.4732 KOps/s 35.0856 KOps/s $\color{#35bf28}+1.10\%$
test_compile_indexing[slice-tensorclass-compile] 0.2047ms 45.0243μs 22.2102 KOps/s 21.9194 KOps/s $\color{#35bf28}+1.33\%$
test_compile_indexing[slice-tensorclass-eager] 0.2689ms 22.6260μs 44.1969 KOps/s 43.2140 KOps/s $\color{#35bf28}+2.27\%$
test_compile_indexing[slice-pytree-compile] 0.1442ms 45.7783μs 21.8444 KOps/s 21.8020 KOps/s $\color{#35bf28}+0.19\%$
test_compile_indexing[slice-pytree-eager] 0.2770ms 22.6554μs 44.1395 KOps/s 43.2159 KOps/s $\color{#35bf28}+2.14\%$
test_compile_indexing[int-tensordict-compile] 97.1610μs 57.6410μs 17.3487 KOps/s 16.9522 KOps/s $\color{#35bf28}+2.34\%$
test_compile_indexing[int-tensordict-eager] 0.2188ms 27.9307μs 35.8029 KOps/s 35.1916 KOps/s $\color{#35bf28}+1.74\%$
test_compile_indexing[int-tensorclass-compile] 0.1449ms 45.5708μs 21.9439 KOps/s 21.9585 KOps/s $\color{#d91a1a}-0.07\%$
test_compile_indexing[int-tensorclass-eager] 0.2713ms 22.7494μs 43.9571 KOps/s 43.3446 KOps/s $\color{#35bf28}+1.41\%$
test_compile_indexing[int-pytree-compile] 81.9710μs 46.1748μs 21.6568 KOps/s 21.8184 KOps/s $\color{#d91a1a}-0.74\%$
test_compile_indexing[int-pytree-eager] 0.2671ms 22.7403μs 43.9748 KOps/s 43.9506 KOps/s $\color{#35bf28}+0.06\%$
test_mod_add[eager] 94.5610μs 50.0511μs 19.9796 KOps/s 19.4490 KOps/s $\color{#35bf28}+2.73\%$
test_mod_add[compile] 0.2129ms 0.1046ms 9.5581 KOps/s 8.9702 KOps/s $\textbf{\color{#35bf28}+6.55\%}$
test_mod_add[compile-overhead] 0.3881ms 0.1492ms 6.7045 KOps/s 6.5738 KOps/s $\color{#35bf28}+1.99\%$
test_mod_wrap[eager] 0.3730ms 0.3004ms 3.3288 KOps/s 3.2414 KOps/s $\color{#35bf28}+2.70\%$
test_mod_wrap[compile] 0.4194ms 0.3628ms 2.7565 KOps/s 2.8261 KOps/s $\color{#d91a1a}-2.46\%$
test_mod_wrap[compile-overhead] 7.3155ms 4.0429ms 247.3472 Ops/s 248.7164 Ops/s $\color{#d91a1a}-0.55\%$
test_mod_wrap_and_backward[eager] 1.6175ms 1.4980ms 667.5396 Ops/s 659.2805 Ops/s $\color{#35bf28}+1.25\%$
test_mod_wrap_and_backward[compile] 1.5529ms 1.4335ms 697.5864 Ops/s 681.9014 Ops/s $\color{#35bf28}+2.30\%$
test_mod_wrap_and_backward[compile-overhead] 1.5871ms 0.8897ms 1.1240 KOps/s 1.1132 KOps/s $\color{#35bf28}+0.97\%$
test_seq_add[eager] 0.2105ms 0.1565ms 6.3883 KOps/s 6.3525 KOps/s $\color{#35bf28}+0.56\%$
test_seq_add[compile] 0.1614ms 0.1194ms 8.3718 KOps/s 8.4091 KOps/s $\color{#d91a1a}-0.44\%$
test_seq_add[compile-overhead] 0.3605ms 0.1584ms 6.3114 KOps/s 6.2455 KOps/s $\color{#35bf28}+1.06\%$
test_seq_wrap[eager] 0.6948ms 0.5246ms 1.9061 KOps/s 1.7922 KOps/s $\textbf{\color{#35bf28}+6.36\%}$
test_seq_wrap[compile] 0.4110ms 0.3674ms 2.7217 KOps/s 2.6658 KOps/s $\color{#35bf28}+2.10\%$
test_seq_wrap[compile-overhead] 0.3287ms 0.2662ms 3.7562 KOps/s 3.5567 KOps/s $\textbf{\color{#35bf28}+5.61\%}$
test_func_call_runtime[False-eager] 0.8972ms 0.8427ms 1.1866 KOps/s 1.1043 KOps/s $\textbf{\color{#35bf28}+7.46\%}$
test_func_call_runtime[False-compile] 0.9822ms 0.9037ms 1.1065 KOps/s 1.0761 KOps/s $\color{#35bf28}+2.83\%$
test_func_call_runtime[False-compile-overhead] 0.5172ms 0.4605ms 2.1717 KOps/s 2.1295 KOps/s $\color{#35bf28}+1.98\%$
test_func_call_runtime[True-eager] 1.1439ms 1.0803ms 925.6623 Ops/s 916.7774 Ops/s $\color{#35bf28}+0.97\%$
test_func_call_runtime[True-compile] 1.0223ms 0.9463ms 1.0567 KOps/s 1.0695 KOps/s $\color{#d91a1a}-1.19\%$
test_func_call_runtime[True-compile-overhead] 0.5442ms 0.4731ms 2.1137 KOps/s 2.0818 KOps/s $\color{#35bf28}+1.53\%$
test_func_call_cm_runtime[False-eager] 0.9119ms 0.8389ms 1.1921 KOps/s 1.1342 KOps/s $\textbf{\color{#35bf28}+5.11\%}$
test_func_call_cm_runtime[False-compile] 1.0300ms 0.9567ms 1.0452 KOps/s 1.0728 KOps/s $\color{#d91a1a}-2.57\%$
test_func_call_cm_runtime[False-compile-overhead] 0.5032ms 0.4623ms 2.1631 KOps/s 2.1354 KOps/s $\color{#35bf28}+1.30\%$
test_func_call_cm_runtime[True-eager] 1.4380ms 1.2279ms 814.4045 Ops/s 804.5845 Ops/s $\color{#35bf28}+1.22\%$
test_func_call_cm_runtime[True-compile] 1.0401ms 0.9527ms 1.0497 KOps/s 1.0357 KOps/s $\color{#35bf28}+1.35\%$
test_func_call_cm_runtime[True-compile-overhead] 0.6114ms 0.5076ms 1.9701 KOps/s 1.9179 KOps/s $\color{#35bf28}+2.72\%$
test_vmap_func_call_cm_runtime[eager] 2.8668ms 2.3712ms 421.7313 Ops/s 415.5892 Ops/s $\color{#35bf28}+1.48\%$
test_vmap_func_call_cm_runtime[compile] 1.0568ms 0.9710ms 1.0299 KOps/s 1.0046 KOps/s $\color{#35bf28}+2.52\%$
test_vmap_func_call_cm_runtime[compile-overhead] 0.5807ms 0.5094ms 1.9633 KOps/s 1.9260 KOps/s $\color{#35bf28}+1.93\%$
test_distributed 2.9074ms 0.1645ms 6.0788 KOps/s 5.9729 KOps/s $\color{#35bf28}+1.77\%$
test_tdmodule 0.2354ms 29.3805μs 34.0361 KOps/s 35.3553 KOps/s $\color{#d91a1a}-3.73\%$
test_tdmodule_dispatch 79.9120μs 45.7809μs 21.8432 KOps/s 21.7631 KOps/s $\color{#35bf28}+0.37\%$
test_tdseq 53.3810μs 26.9792μs 37.0656 KOps/s 37.0278 KOps/s $\color{#35bf28}+0.10\%$
test_tdseq_dispatch 75.9310μs 47.3129μs 21.1359 KOps/s 20.9766 KOps/s $\color{#35bf28}+0.76\%$
test_instantiation_functorch 2.2127ms 2.0660ms 484.0230 Ops/s 481.8181 Ops/s $\color{#35bf28}+0.46\%$
test_exec_functorch 0.2549ms 0.1923ms 5.2001 KOps/s 5.5076 KOps/s $\textbf{\color{#d91a1a}-5.58\%}$
test_exec_functional_call 0.2392ms 0.1772ms 5.6440 KOps/s 6.1772 KOps/s $\textbf{\color{#d91a1a}-8.63\%}$
test_exec_td_decorator 0.4765ms 0.2526ms 3.9582 KOps/s 4.2030 KOps/s $\textbf{\color{#d91a1a}-5.83\%}$
test_vmap_mlp_speed_decorator[True-True] 1.0670ms 0.8516ms 1.1743 KOps/s 1.2010 KOps/s $\color{#d91a1a}-2.22\%$
test_vmap_mlp_speed_decorator[True-False] 1.0254ms 0.8486ms 1.1784 KOps/s 1.2026 KOps/s $\color{#d91a1a}-2.01\%$
test_vmap_mlp_speed_decorator[False-True] 0.9127ms 0.7334ms 1.3635 KOps/s 1.3885 KOps/s $\color{#d91a1a}-1.80\%$
test_vmap_mlp_speed_decorator[False-False] 0.9330ms 0.7458ms 1.3408 KOps/s 1.3861 KOps/s $\color{#d91a1a}-3.27\%$
test_vmap_transformer_speed_decorator[True-True] 20.7368ms 20.5826ms 48.5848 Ops/s 48.3328 Ops/s $\color{#35bf28}+0.52\%$
test_vmap_transformer_speed_decorator[True-False] 21.1628ms 20.6217ms 48.4927 Ops/s 48.3214 Ops/s $\color{#35bf28}+0.35\%$
test_vmap_transformer_speed_decorator[False-True] 20.8930ms 20.4782ms 48.8323 Ops/s 48.7254 Ops/s $\color{#35bf28}+0.22\%$
test_vmap_transformer_speed_decorator[False-False] 21.5992ms 20.4598ms 48.8762 Ops/s 48.7433 Ops/s $\color{#35bf28}+0.27\%$
test_to_module_speed[True] 2.0736ms 1.4737ms 678.5584 Ops/s 669.4749 Ops/s $\color{#35bf28}+1.36\%$
test_to_module_speed[False] 1.5197ms 1.4377ms 695.5585 Ops/s 685.7052 Ops/s $\color{#35bf28}+1.44\%$
test_tc_init 70.7120μs 45.7095μs 21.8773 KOps/s 21.3120 KOps/s $\color{#35bf28}+2.65\%$
test_tc_init_tensor_only 38.0000μs 10.0071μs 99.9290 KOps/s 102.0868 KOps/s $\color{#d91a1a}-2.11\%$
test_tc_init_nested 0.1376ms 90.0471μs 11.1053 KOps/s 10.7026 KOps/s $\color{#35bf28}+3.76\%$
test_tc_init_many_fields 50.6110μs 16.5135μs 60.5564 KOps/s 60.2936 KOps/s $\color{#35bf28}+0.44\%$
test_tc_first_layer_tensor 22.1700μs 1.8668μs 535.6792 KOps/s 535.1108 KOps/s $\color{#35bf28}+0.11\%$
test_tc_first_layer_tensor_only 5.8216μs 0.7586μs 1.3183 MOps/s 1.3189 MOps/s $\color{#d91a1a}-0.05\%$
test_tc_first_layer_tensor_set 32.5700μs 3.9568μs 252.7323 KOps/s 247.5837 KOps/s $\color{#35bf28}+2.08\%$
test_tc_first_layer_tensor_only_set 28.9300μs 3.3820μs 295.6804 KOps/s 317.3433 KOps/s $\textbf{\color{#d91a1a}-6.83\%}$
test_tc_first_layer_nontensor 30.7110μs 6.2263μs 160.6100 KOps/s 160.6045 KOps/s $+0.00\%$
test_tc_second_layer_tensor 56.8710μs 4.3948μs 227.5441 KOps/s 226.4866 KOps/s $\color{#35bf28}+0.47\%$
test_tc_second_layer_nontensor 37.5200μs 8.8843μs 112.5582 KOps/s 115.6201 KOps/s $\color{#d91a1a}-2.65\%$
test_unbind 0.2488s 16.2521ms 61.5304 Ops/s 66.7208 Ops/s $\textbf{\color{#d91a1a}-7.78\%}$
test_full_like 4.7647ms 4.3731ms 228.6700 Ops/s 228.1209 Ops/s $\color{#35bf28}+0.24\%$
test_zeros_like 4.4621ms 4.3581ms 229.4552 Ops/s 228.9247 Ops/s $\color{#35bf28}+0.23\%$
test_ones_like 5.5122ms 4.3681ms 228.9346 Ops/s 228.5067 Ops/s $\color{#35bf28}+0.19\%$
test_clone 6.6406ms 6.4200ms 155.7631 Ops/s 154.4666 Ops/s $\color{#35bf28}+0.84\%$
test_squeeze 77.7710μs 15.4177μs 64.8606 KOps/s 70.1963 KOps/s $\textbf{\color{#d91a1a}-7.60\%}$
test_unsqueeze 0.1720ms 0.1157ms 8.6441 KOps/s 9.0967 KOps/s $\color{#d91a1a}-4.98\%$
test_split 0.3726ms 0.1847ms 5.4141 KOps/s 5.2471 KOps/s $\color{#35bf28}+3.18\%$
test_permute 0.2971ms 0.2041ms 4.8985 KOps/s 4.9440 KOps/s $\color{#d91a1a}-0.92\%$
test_stack 51.6973ms 51.1901ms 19.5350 Ops/s 19.5566 Ops/s $\color{#d91a1a}-0.11\%$
test_cat 51.4864ms 51.1573ms 19.5476 Ops/s 23.3409 Ops/s $\textbf{\color{#d91a1a}-16.25\%}$

@github-actions

github-actions Bot commented Feb 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 243. Improved: $\large\color{#35bf28}29$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 46.1900μs 13.9742μs 71.5606 KOps/s 69.7754 KOps/s $\color{#35bf28}+2.56\%$
test_plain_set_stack_nested 42.0510μs 14.2850μs 70.0035 KOps/s 69.3630 KOps/s $\color{#35bf28}+0.92\%$
test_plain_set_nested_inplace 53.3110μs 15.8265μs 63.1850 KOps/s 62.8418 KOps/s $\color{#35bf28}+0.55\%$
test_plain_set_stack_nested_inplace 44.0110μs 15.5830μs 64.1726 KOps/s 62.8889 KOps/s $\color{#35bf28}+2.04\%$
test_items 69.8110μs 5.5110μs 181.4539 KOps/s 179.9938 KOps/s $\color{#35bf28}+0.81\%$
test_items_nested 0.5255ms 0.4345ms 2.3014 KOps/s 2.2705 KOps/s $\color{#35bf28}+1.36\%$
test_items_nested_locked 0.6440ms 0.4349ms 2.2996 KOps/s 2.2605 KOps/s $\color{#35bf28}+1.73\%$
test_items_nested_leaf 0.1229ms 91.7740μs 10.8963 KOps/s 10.7074 KOps/s $\color{#35bf28}+1.76\%$
test_items_stack_nested 0.5090ms 0.4350ms 2.2991 KOps/s 2.2506 KOps/s $\color{#35bf28}+2.15\%$
test_items_stack_nested_leaf 0.1337ms 92.6663μs 10.7914 KOps/s 10.7489 KOps/s $\color{#35bf28}+0.40\%$
test_items_stack_nested_locked 0.4922ms 0.4404ms 2.2706 KOps/s 2.2574 KOps/s $\color{#35bf28}+0.58\%$
test_keys 30.0200μs 4.1128μs 243.1423 KOps/s 230.0972 KOps/s $\textbf{\color{#35bf28}+5.67\%}$
test_keys_nested 0.1627ms 0.1172ms 8.5299 KOps/s 8.4329 KOps/s $\color{#35bf28}+1.15\%$
test_keys_nested_locked 0.7490ms 0.1251ms 7.9911 KOps/s 7.8599 KOps/s $\color{#35bf28}+1.67\%$
test_keys_nested_leaf 0.1549ms 0.1073ms 9.3205 KOps/s 9.1302 KOps/s $\color{#35bf28}+2.09\%$
test_keys_stack_nested 0.1650ms 0.1180ms 8.4777 KOps/s 8.4797 KOps/s $\color{#d91a1a}-0.02\%$
test_keys_stack_nested_leaf 0.1586ms 0.1082ms 9.2461 KOps/s 9.1238 KOps/s $\color{#35bf28}+1.34\%$
test_keys_stack_nested_locked 0.1707ms 0.1247ms 8.0199 KOps/s 7.8997 KOps/s $\color{#35bf28}+1.52\%$
test_values 7.3002μs 0.9980μs 1.0020 MOps/s 1.0041 MOps/s $\color{#d91a1a}-0.21\%$
test_values_nested 78.8120μs 46.3609μs 21.5699 KOps/s 21.1098 KOps/s $\color{#35bf28}+2.18\%$
test_values_nested_locked 96.9710μs 49.2892μs 20.2884 KOps/s 19.9203 KOps/s $\color{#35bf28}+1.85\%$
test_values_nested_leaf 85.4410μs 52.8934μs 18.9060 KOps/s 18.6874 KOps/s $\color{#35bf28}+1.17\%$
test_values_stack_nested 87.4310μs 46.3894μs 21.5567 KOps/s 21.3062 KOps/s $\color{#35bf28}+1.18\%$
test_values_stack_nested_leaf 0.1112ms 52.8311μs 18.9283 KOps/s 18.7567 KOps/s $\color{#35bf28}+0.91\%$
test_values_stack_nested_locked 91.1710μs 49.0898μs 20.3708 KOps/s 19.8638 KOps/s $\color{#35bf28}+2.55\%$
test_membership 6.1118μs 0.7971μs 1.2546 MOps/s 1.2228 MOps/s $\color{#35bf28}+2.61\%$
test_membership_nested 28.2110μs 2.7521μs 363.3653 KOps/s 360.2740 KOps/s $\color{#35bf28}+0.86\%$
test_membership_nested_leaf 20.9955μs 2.6561μs 376.4866 KOps/s 357.0050 KOps/s $\textbf{\color{#35bf28}+5.46\%}$
test_membership_stacked_nested 31.5810μs 2.7506μs 363.5572 KOps/s 359.6684 KOps/s $\color{#35bf28}+1.08\%$
test_membership_stacked_nested_leaf 31.8810μs 2.7813μs 359.5407 KOps/s 356.3172 KOps/s $\color{#35bf28}+0.90\%$
test_membership_nested_last 38.9310μs 4.1075μs 243.4547 KOps/s 238.3851 KOps/s $\color{#35bf28}+2.13\%$
test_membership_nested_leaf_last 30.9400μs 4.1311μs 242.0678 KOps/s 238.3257 KOps/s $\color{#35bf28}+1.57\%$
test_membership_stacked_nested_last 58.2910μs 4.1091μs 243.3630 KOps/s 240.9635 KOps/s $\color{#35bf28}+1.00\%$
test_membership_stacked_nested_leaf_last 39.4910μs 4.1308μs 242.0825 KOps/s 240.4629 KOps/s $\color{#35bf28}+0.67\%$
test_nested_getleaf 53.1910μs 20.4538μs 48.8907 KOps/s 48.2983 KOps/s $\color{#35bf28}+1.23\%$
test_nested_get 46.1110μs 19.0480μs 52.4988 KOps/s 51.0342 KOps/s $\color{#35bf28}+2.87\%$
test_stacked_getleaf 52.7510μs 20.5232μs 48.7255 KOps/s 49.0271 KOps/s $\color{#d91a1a}-0.62\%$
test_stacked_get 46.9910μs 19.1971μs 52.0911 KOps/s 49.3608 KOps/s $\textbf{\color{#35bf28}+5.53\%}$
test_nested_getitemleaf 50.4910μs 21.1761μs 47.2230 KOps/s 47.7134 KOps/s $\color{#d91a1a}-1.03\%$
test_nested_getitem 53.2910μs 19.6881μs 50.7922 KOps/s 50.1884 KOps/s $\color{#35bf28}+1.20\%$
test_stacked_getitemleaf 63.9610μs 20.6944μs 48.3223 KOps/s 47.1804 KOps/s $\color{#35bf28}+2.42\%$
test_stacked_getitem 51.8210μs 19.4942μs 51.2974 KOps/s 50.7662 KOps/s $\color{#35bf28}+1.05\%$
test_lock_nested 0.5622ms 0.4505ms 2.2199 KOps/s 2.1934 KOps/s $\color{#35bf28}+1.21\%$
test_lock_stack_nested 0.5155ms 0.4543ms 2.2010 KOps/s 2.1598 KOps/s $\color{#35bf28}+1.91\%$
test_unlock_nested 0.4425ms 0.3658ms 2.7340 KOps/s 2.7251 KOps/s $\color{#35bf28}+0.33\%$
test_unlock_stack_nested 0.5136ms 0.3660ms 2.7326 KOps/s 2.6755 KOps/s $\color{#35bf28}+2.13\%$
test_flatten_speed 0.2544ms 0.1165ms 8.5855 KOps/s 8.4592 KOps/s $\color{#35bf28}+1.49\%$
test_unflatten_speed 0.5678ms 0.5427ms 1.8425 KOps/s 1.8002 KOps/s $\color{#35bf28}+2.35\%$
test_common_ops 0.8101ms 0.6611ms 1.5126 KOps/s 1.4777 KOps/s $\color{#35bf28}+2.36\%$
test_creation 51.8110μs 2.7388μs 365.1190 KOps/s 365.9800 KOps/s $\color{#d91a1a}-0.24\%$
test_creation_empty 25.5610μs 5.7772μs 173.0938 KOps/s 172.3501 KOps/s $\color{#35bf28}+0.43\%$
test_creation_nested_1 36.4500μs 9.8846μs 101.1674 KOps/s 100.4523 KOps/s $\color{#35bf28}+0.71\%$
test_creation_nested_2 36.9000μs 11.1966μs 89.3129 KOps/s 89.1708 KOps/s $\color{#35bf28}+0.16\%$
test_creation_many_keys[10] 50.1210μs 16.9878μs 58.8658 KOps/s 59.0016 KOps/s $\color{#d91a1a}-0.23\%$
test_creation_many_keys[50] 0.1032ms 73.3693μs 13.6297 KOps/s 13.9038 KOps/s $\color{#d91a1a}-1.97\%$
test_creation_many_keys[100] 0.1783ms 0.1440ms 6.9461 KOps/s 7.1037 KOps/s $\color{#d91a1a}-2.22\%$
test_creation_nested_many_keys[10] 66.8010μs 36.9231μs 27.0833 KOps/s 27.9177 KOps/s $\color{#d91a1a}-2.99\%$
test_creation_nested_many_keys[50] 0.2068ms 0.1503ms 6.6526 KOps/s 6.8235 KOps/s $\color{#d91a1a}-2.51\%$
test_clone 43.5410μs 12.7207μs 78.6119 KOps/s 75.6078 KOps/s $\color{#35bf28}+3.97\%$
test_getitem[int] 1.8261ms 14.0392μs 71.2292 KOps/s 57.5103 KOps/s $\textbf{\color{#35bf28}+23.85\%}$
test_getitem[slice_int] 0.1405ms 23.5822μs 42.4049 KOps/s 42.0627 KOps/s $\color{#35bf28}+0.81\%$
test_getitem[range] 0.1919ms 59.5604μs 16.7897 KOps/s 16.2006 KOps/s $\color{#35bf28}+3.64\%$
test_getitem[tuple] 0.1374ms 22.9560μs 43.5616 KOps/s 43.0289 KOps/s $\color{#35bf28}+1.24\%$
test_getitem[list] 0.1793ms 54.1040μs 18.4829 KOps/s 16.8436 KOps/s $\textbf{\color{#35bf28}+9.73\%}$
test_setitem_dim[int] 53.0410μs 24.6290μs 40.6025 KOps/s 37.7382 KOps/s $\textbf{\color{#35bf28}+7.59\%}$
test_setitem_dim[slice_int] 79.5210μs 42.1115μs 23.7465 KOps/s 22.5470 KOps/s $\textbf{\color{#35bf28}+5.32\%}$
test_setitem_dim[range] 0.1202ms 90.4894μs 11.0510 KOps/s 10.9669 KOps/s $\color{#35bf28}+0.77\%$
test_setitem_dim[tuple] 60.9910μs 38.8322μs 25.7518 KOps/s 24.6787 KOps/s $\color{#35bf28}+4.35\%$
test_setitem 45.7710μs 16.5710μs 60.3465 KOps/s 57.2527 KOps/s $\textbf{\color{#35bf28}+5.40\%}$
test_set 61.8500μs 16.0666μs 62.2408 KOps/s 59.4099 KOps/s $\color{#35bf28}+4.77\%$
test_set_shared 0.7506ms 0.2062ms 4.8502 KOps/s 4.7432 KOps/s $\color{#35bf28}+2.26\%$
test_update 0.2811ms 21.6669μs 46.1533 KOps/s 47.2509 KOps/s $\color{#d91a1a}-2.32\%$
test_update_nested 89.3520μs 33.1176μs 30.1954 KOps/s 30.6409 KOps/s $\color{#d91a1a}-1.45\%$
test_update__nested 0.4721ms 33.2567μs 30.0691 KOps/s 29.2385 KOps/s $\color{#35bf28}+2.84\%$
test_set_nested 58.6410μs 19.5658μs 51.1097 KOps/s 53.6849 KOps/s $\color{#d91a1a}-4.80\%$
test_set_nested_new 59.5110μs 23.3132μs 42.8941 KOps/s 42.5624 KOps/s $\color{#35bf28}+0.78\%$
test_select 75.2710μs 40.2891μs 24.8206 KOps/s 24.4916 KOps/s $\color{#35bf28}+1.34\%$
test_select_nested 0.1102ms 70.0661μs 14.2722 KOps/s 14.0297 KOps/s $\color{#35bf28}+1.73\%$
test_exclude_nested 0.1204ms 86.1911μs 11.6021 KOps/s 11.5080 KOps/s $\color{#35bf28}+0.82\%$
test_empty[True] 0.4684ms 0.3702ms 2.7012 KOps/s 2.6400 KOps/s $\color{#35bf28}+2.32\%$
test_empty[False] 8.9675μs 1.2536μs 797.6989 KOps/s 791.9975 KOps/s $\color{#35bf28}+0.72\%$
test_to 0.1014ms 69.2294μs 14.4447 KOps/s 14.0152 KOps/s $\color{#35bf28}+3.06\%$
test_to_nonblocking 0.1134ms 63.4956μs 15.7491 KOps/s 15.9853 KOps/s $\color{#d91a1a}-1.48\%$
test_unbind_speed 0.3400ms 0.3146ms 3.1791 KOps/s 3.2291 KOps/s $\color{#d91a1a}-1.55\%$
test_unbind_speed_stack0 0.3529ms 0.3114ms 3.2108 KOps/s 3.2198 KOps/s $\color{#d91a1a}-0.28\%$
test_unbind_speed_stack1 0.1037s 0.8062ms 1.2404 KOps/s 1.1235 KOps/s $\textbf{\color{#35bf28}+10.41\%}$
test_split 0.1034s 1.2109ms 825.8411 Ops/s 912.1105 Ops/s $\textbf{\color{#d91a1a}-9.46\%}$
test_chunk 0.1035s 1.1646ms 858.6373 Ops/s 954.1312 Ops/s $\textbf{\color{#d91a1a}-10.01\%}$
test_to_cpu_blocking 19.0507ms 18.3202ms 54.5845 Ops/s 49.1888 Ops/s $\textbf{\color{#35bf28}+10.97\%}$
test_to_cpu_global_sync 10.8937ms 10.7396ms 93.1130 Ops/s 91.9211 Ops/s $\color{#35bf28}+1.30\%$
test_to_cpu_event_sync 11.8448ms 11.5396ms 86.6582 Ops/s 85.6176 Ops/s $\color{#35bf28}+1.22\%$
test_to_cpu_default 0.1149s 12.7528ms 78.4140 Ops/s 85.7163 Ops/s $\textbf{\color{#d91a1a}-8.52\%}$
test_consolidate[False-None] 4.1313ms 3.9480ms 253.2937 Ops/s 225.9166 Ops/s $\textbf{\color{#35bf28}+12.12\%}$
test_consolidate[default-None] 1.9861ms 1.9082ms 524.0541 Ops/s 496.9723 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_consolidate[reduce-overhead-None] 1.9264ms 1.8498ms 540.6134 Ops/s 519.2570 Ops/s $\color{#35bf28}+4.11\%$
test_consolidate_njt[False-None] 8.4800ms 8.1382ms 122.8778 Ops/s 120.6505 Ops/s $\color{#35bf28}+1.85\%$
test_to[False-False-None] 2.1000ms 1.9966ms 500.8596 Ops/s 492.6865 Ops/s $\color{#35bf28}+1.66\%$
test_to[True-False-None] 2.1341ms 1.8805ms 531.7623 Ops/s 516.0775 Ops/s $\color{#35bf28}+3.04\%$
test_to[within-False-None] 6.1813ms 5.8984ms 169.5376 Ops/s 166.2252 Ops/s $\color{#35bf28}+1.99\%$
test_to[True-default-None] 7.7387ms 7.3636ms 135.8030 Ops/s 130.2636 Ops/s $\color{#35bf28}+4.25\%$
test_to_njt[False-False-None] 8.2914ms 8.1808ms 122.2375 Ops/s 120.2395 Ops/s $\color{#35bf28}+1.66\%$
test_to_njt[True-False-None] 6.8435ms 6.7049ms 149.1447 Ops/s 143.9412 Ops/s $\color{#35bf28}+3.62\%$
test_to_njt[within-False-None] 14.9782ms 14.9087ms 67.0748 Ops/s 64.5001 Ops/s $\color{#35bf28}+3.99\%$
test_creation[device0] 0.4740ms 0.1179ms 8.4848 KOps/s 8.5095 KOps/s $\color{#d91a1a}-0.29\%$
test_creation_from_tensor 0.4906ms 0.1117ms 8.9515 KOps/s 8.7151 KOps/s $\color{#35bf28}+2.71\%$
test_add_one[memmap_tensor0] 0.3695ms 6.3377μs 157.7865 KOps/s 158.2864 KOps/s $\color{#d91a1a}-0.32\%$
test_contiguous[memmap_tensor0] 28.0310μs 0.6056μs 1.6513 MOps/s 2.3085 MOps/s $\textbf{\color{#d91a1a}-28.47\%}$
test_stack[memmap_tensor0] 27.1410μs 4.4366μs 225.3996 KOps/s 220.5803 KOps/s $\color{#35bf28}+2.18\%$
test_memmaptd_index 1.0837ms 0.2522ms 3.9655 KOps/s 3.9060 KOps/s $\color{#35bf28}+1.52\%$
test_memmaptd_index_astensor 0.4975ms 0.3405ms 2.9369 KOps/s 2.8996 KOps/s $\color{#35bf28}+1.28\%$
test_memmaptd_index_op 0.8359ms 0.5825ms 1.7169 KOps/s 1.6814 KOps/s $\color{#35bf28}+2.11\%$
test_serialize_model 0.1391s 0.1362s 7.3447 Ops/s 7.3106 Ops/s $\color{#35bf28}+0.47\%$
test_serialize_model_pickle 1.3481s 1.2096s 0.8267 Ops/s 0.8196 Ops/s $\color{#35bf28}+0.87\%$
test_serialize_weights 0.1393s 0.1356s 7.3746 Ops/s 7.3758 Ops/s $\color{#d91a1a}-0.02\%$
test_serialize_weights_returnearly 0.4178s 91.7951ms 10.8938 Ops/s 10.1962 Ops/s $\textbf{\color{#35bf28}+6.84\%}$
test_serialize_weights_pickle 1.3644s 1.1965s 0.8357 Ops/s 0.8126 Ops/s $\color{#35bf28}+2.85\%$
test_reshape_pytree 0.2159ms 32.4067μs 30.8579 KOps/s 30.4517 KOps/s $\color{#35bf28}+1.33\%$
test_reshape_td 71.1210μs 43.9741μs 22.7407 KOps/s 23.2956 KOps/s $\color{#d91a1a}-2.38\%$
test_view_pytree 0.2082ms 32.0438μs 31.2073 KOps/s 30.8912 KOps/s $\color{#35bf28}+1.02\%$
test_view_td 99.1420μs 53.0893μs 18.8362 KOps/s 19.7245 KOps/s $\color{#d91a1a}-4.50\%$
test_unbind_pytree 0.2488ms 35.8436μs 27.8990 KOps/s 28.0465 KOps/s $\color{#d91a1a}-0.53\%$
test_unbind_td 88.2020μs 47.1100μs 21.2269 KOps/s 21.4185 KOps/s $\color{#d91a1a}-0.89\%$
test_split_pytree 0.2497ms 40.9195μs 24.4383 KOps/s 24.3247 KOps/s $\color{#35bf28}+0.47\%$
test_split_td 0.1132ms 62.5596μs 15.9848 KOps/s 16.0765 KOps/s $\color{#d91a1a}-0.57\%$
test_add_pytree 0.2225ms 40.7893μs 24.5162 KOps/s 23.7539 KOps/s $\color{#35bf28}+3.21\%$
test_add_td 97.9710μs 52.0392μs 19.2163 KOps/s 18.2840 KOps/s $\textbf{\color{#35bf28}+5.10\%}$
test_compile_add_one_nested[tensordict-compile] 0.2030ms 0.1356ms 7.3762 KOps/s 6.9440 KOps/s $\textbf{\color{#35bf28}+6.22\%}$
test_compile_add_one_nested[tensordict-eager] 0.3229ms 0.1877ms 5.3280 KOps/s 5.4362 KOps/s $\color{#d91a1a}-1.99\%$
test_compile_add_one_nested[pytree-compile] 0.1794ms 0.1063ms 9.4054 KOps/s 9.2767 KOps/s $\color{#35bf28}+1.39\%$
test_compile_add_one_nested[pytree-eager] 0.4224ms 0.1724ms 5.8010 KOps/s 5.7442 KOps/s $\color{#35bf28}+0.99\%$
test_compile_copy_nested[tensordict-compile] 87.1810μs 29.2548μs 34.1824 KOps/s 30.7386 KOps/s $\textbf{\color{#35bf28}+11.20\%}$
test_compile_copy_nested[tensordict-eager] 0.1077ms 49.6314μs 20.1486 KOps/s 20.0374 KOps/s $\color{#35bf28}+0.55\%$
test_compile_copy_nested[pytree-compile] 49.0000μs 9.4771μs 105.5179 KOps/s 102.4685 KOps/s $\color{#35bf28}+2.98\%$
test_compile_copy_nested[pytree-eager] 0.4745ms 66.3023μs 15.0824 KOps/s 14.9860 KOps/s $\color{#35bf28}+0.64\%$
test_compile_add_one_flat[tensordict-compile] 0.2231ms 0.1732ms 5.7742 KOps/s 5.6155 KOps/s $\color{#35bf28}+2.83\%$
test_compile_add_one_flat[tensordict-eager] 0.3380ms 0.2490ms 4.0165 KOps/s 3.9966 KOps/s $\color{#35bf28}+0.50\%$
test_compile_add_one_flat[tensorclass-compile] 0.1940ms 0.1143ms 8.7484 KOps/s 8.1450 KOps/s $\textbf{\color{#35bf28}+7.41\%}$
test_compile_add_one_flat[tensorclass-eager] 0.1138ms 69.3067μs 14.4286 KOps/s 13.9440 KOps/s $\color{#35bf28}+3.48\%$
test_compile_add_one_flat[pytree-compile] 0.2050ms 0.1556ms 6.4269 KOps/s 6.0244 KOps/s $\textbf{\color{#35bf28}+6.68\%}$
test_compile_add_one_flat[pytree-eager] 0.9336ms 0.5436ms 1.8396 KOps/s 1.9031 KOps/s $\color{#d91a1a}-3.34\%$
test_compile_add_self_flat[tensordict-eager] 0.3662ms 0.3022ms 3.3088 KOps/s 3.2261 KOps/s $\color{#35bf28}+2.56\%$
test_compile_add_self_flat[tensordict-compile] 0.2334ms 0.1765ms 5.6643 KOps/s 5.2646 KOps/s $\textbf{\color{#35bf28}+7.59\%}$
test_compile_add_self_flat[tensorclass-eager] 0.1299ms 83.5050μs 11.9753 KOps/s 11.6000 KOps/s $\color{#35bf28}+3.24\%$
test_compile_add_self_flat[tensorclass-compile] 0.1630ms 0.1229ms 8.1363 KOps/s 8.1394 KOps/s $\color{#d91a1a}-0.04\%$
test_compile_add_self_flat[pytree-eager] 0.6703ms 0.4234ms 2.3620 KOps/s 2.2828 KOps/s $\color{#35bf28}+3.47\%$
test_compile_add_self_flat[pytree-compile] 0.2155ms 0.1564ms 6.3932 KOps/s 6.1395 KOps/s $\color{#35bf28}+4.13\%$
test_compile_copy_flat[tensordict-compile] 0.2440ms 23.9058μs 41.8308 KOps/s 37.3395 KOps/s $\textbf{\color{#35bf28}+12.03\%}$
test_compile_copy_flat[tensordict-eager] 78.7410μs 39.7342μs 25.1672 KOps/s 24.9528 KOps/s $\color{#35bf28}+0.86\%$
test_compile_copy_flat[pytree-compile] 46.0800μs 10.4161μs 96.0050 KOps/s 89.0040 KOps/s $\textbf{\color{#35bf28}+7.87\%}$
test_compile_copy_flat[pytree-eager] 0.4175ms 51.1108μs 19.5653 KOps/s 19.5512 KOps/s $\color{#35bf28}+0.07\%$
test_compile_assign_and_add[tensordict-compile] 1.9231ms 0.1690ms 5.9179 KOps/s 5.6175 KOps/s $\textbf{\color{#35bf28}+5.35\%}$
test_compile_assign_and_add[tensordict-eager] 3.3391ms 3.1884ms 313.6388 Ops/s 307.0882 Ops/s $\color{#35bf28}+2.13\%$
test_compile_assign_and_add[pytree-compile] 1.9086ms 0.1574ms 6.3535 KOps/s 6.2488 KOps/s $\color{#35bf28}+1.68\%$
test_compile_assign_and_add[pytree-eager] 2.9068ms 2.7036ms 369.8776 Ops/s 368.7820 Ops/s $\color{#35bf28}+0.30\%$
test_compile_indexing[tensor-tensordict-compile] 0.1558ms 0.1062ms 9.4201 KOps/s 9.1165 KOps/s $\color{#35bf28}+3.33\%$
test_compile_indexing[tensor-tensordict-eager] 0.3152ms 73.5196μs 13.6018 KOps/s 13.9313 KOps/s $\color{#d91a1a}-2.36\%$
test_compile_indexing[tensor-tensorclass-compile] 0.1512ms 95.6186μs 10.4582 KOps/s 10.2347 KOps/s $\color{#35bf28}+2.18\%$
test_compile_indexing[tensor-tensorclass-eager] 0.2487ms 43.1273μs 23.1872 KOps/s 21.0659 KOps/s $\textbf{\color{#35bf28}+10.07\%}$
test_compile_indexing[tensor-pytree-compile] 0.1549ms 94.7023μs 10.5594 KOps/s 10.1855 KOps/s $\color{#35bf28}+3.67\%$
test_compile_indexing[tensor-pytree-eager] 0.2464ms 43.0745μs 23.2156 KOps/s 21.1433 KOps/s $\textbf{\color{#35bf28}+9.80\%}$
test_compile_indexing[slice-tensordict-compile] 0.1036ms 55.4612μs 18.0306 KOps/s 17.4642 KOps/s $\color{#35bf28}+3.24\%$
test_compile_indexing[slice-tensordict-eager] 0.2226ms 27.2232μs 36.7333 KOps/s 36.8455 KOps/s $\color{#d91a1a}-0.30\%$
test_compile_indexing[slice-tensorclass-compile] 0.1642ms 45.1378μs 22.1544 KOps/s 22.2655 KOps/s $\color{#d91a1a}-0.50\%$
test_compile_indexing[slice-tensorclass-eager] 0.2564ms 21.6528μs 46.1834 KOps/s 45.4923 KOps/s $\color{#35bf28}+1.52\%$
test_compile_indexing[slice-pytree-compile] 86.4610μs 44.9229μs 22.2603 KOps/s 21.7525 KOps/s $\color{#35bf28}+2.33\%$
test_compile_indexing[slice-pytree-eager] 0.2551ms 21.7581μs 45.9600 KOps/s 45.6150 KOps/s $\color{#35bf28}+0.76\%$
test_compile_indexing[int-tensordict-compile] 0.1017ms 55.5523μs 18.0011 KOps/s 17.8485 KOps/s $\color{#35bf28}+0.85\%$
test_compile_indexing[int-tensordict-eager] 0.2116ms 26.8864μs 37.1935 KOps/s 37.0112 KOps/s $\color{#35bf28}+0.49\%$
test_compile_indexing[int-tensorclass-compile] 89.3220μs 45.3072μs 22.0716 KOps/s 21.9538 KOps/s $\color{#35bf28}+0.54\%$
test_compile_indexing[int-tensorclass-eager] 0.2680ms 21.6634μs 46.1608 KOps/s 45.4077 KOps/s $\color{#35bf28}+1.66\%$
test_compile_indexing[int-pytree-compile] 81.8710μs 45.0808μs 22.1824 KOps/s 21.9809 KOps/s $\color{#35bf28}+0.92\%$
test_compile_indexing[int-pytree-eager] 0.2504ms 21.6044μs 46.2868 KOps/s 45.4122 KOps/s $\color{#35bf28}+1.93\%$
test_mod_add[eager] 0.1116ms 48.3643μs 20.6764 KOps/s 20.6625 KOps/s $\color{#35bf28}+0.07\%$
test_mod_add[compile] 0.1798ms 0.1023ms 9.7744 KOps/s 9.7004 KOps/s $\color{#35bf28}+0.76\%$
test_mod_add[compile-overhead] 0.2281ms 0.1459ms 6.8549 KOps/s 6.8527 KOps/s $\color{#35bf28}+0.03\%$
test_mod_wrap[eager] 0.3564ms 0.3003ms 3.3296 KOps/s 3.4957 KOps/s $\color{#d91a1a}-4.75\%$
test_mod_wrap[compile] 0.4107ms 0.3522ms 2.8394 KOps/s 2.9305 KOps/s $\color{#d91a1a}-3.11\%$
test_mod_wrap[compile-overhead] 7.3155ms 4.0369ms 247.7120 Ops/s 252.0003 Ops/s $\color{#d91a1a}-1.70\%$
test_mod_wrap_and_backward[eager] 1.6621ms 1.5012ms 666.1542 Ops/s 676.2642 Ops/s $\color{#d91a1a}-1.49\%$
test_mod_wrap_and_backward[compile] 1.6106ms 1.4117ms 708.3629 Ops/s 703.0652 Ops/s $\color{#35bf28}+0.75\%$
test_mod_wrap_and_backward[compile-overhead] 1.2181ms 0.8643ms 1.1570 KOps/s 1.1372 KOps/s $\color{#35bf28}+1.74\%$
test_seq_add[eager] 0.2260ms 0.1560ms 6.4119 KOps/s 6.6085 KOps/s $\color{#d91a1a}-2.97\%$
test_seq_add[compile] 0.2243ms 0.1158ms 8.6344 KOps/s 8.5398 KOps/s $\color{#35bf28}+1.11\%$
test_seq_add[compile-overhead] 0.3723ms 0.1576ms 6.3457 KOps/s 6.4952 KOps/s $\color{#d91a1a}-2.30\%$
test_seq_wrap[eager] 0.6265ms 0.5403ms 1.8507 KOps/s 1.9576 KOps/s $\textbf{\color{#d91a1a}-5.46\%}$
test_seq_wrap[compile] 0.5152ms 0.3581ms 2.7926 KOps/s 2.7820 KOps/s $\color{#35bf28}+0.38\%$
test_seq_wrap[compile-overhead] 0.3424ms 0.2623ms 3.8129 KOps/s 3.8316 KOps/s $\color{#d91a1a}-0.49\%$
test_func_call_runtime[False-eager] 0.9428ms 0.8496ms 1.1770 KOps/s 1.2179 KOps/s $\color{#d91a1a}-3.35\%$
test_func_call_runtime[False-compile] 0.9335ms 0.8784ms 1.1385 KOps/s 1.1152 KOps/s $\color{#35bf28}+2.09\%$
test_func_call_runtime[False-compile-overhead] 0.5033ms 0.4437ms 2.2536 KOps/s 2.2251 KOps/s $\color{#35bf28}+1.28\%$
test_func_call_runtime[True-eager] 1.1711ms 1.0591ms 944.1680 Ops/s 940.2324 Ops/s $\color{#35bf28}+0.42\%$
test_func_call_runtime[True-compile] 0.9668ms 0.8898ms 1.1239 KOps/s 1.1120 KOps/s $\color{#35bf28}+1.07\%$
test_func_call_runtime[True-compile-overhead] 0.5055ms 0.4583ms 2.1819 KOps/s 2.1531 KOps/s $\color{#35bf28}+1.34\%$
test_func_call_cm_runtime[False-eager] 0.8826ms 0.8217ms 1.2170 KOps/s 1.1501 KOps/s $\textbf{\color{#35bf28}+5.81\%}$
test_func_call_cm_runtime[False-compile] 1.1248ms 0.9236ms 1.0828 KOps/s 1.1256 KOps/s $\color{#d91a1a}-3.80\%$
test_func_call_cm_runtime[False-compile-overhead] 0.5363ms 0.4470ms 2.2372 KOps/s 2.2161 KOps/s $\color{#35bf28}+0.95\%$
test_func_call_cm_runtime[True-eager] 1.2873ms 1.1920ms 838.9229 Ops/s 829.5009 Ops/s $\color{#35bf28}+1.14\%$
test_func_call_cm_runtime[True-compile] 1.0173ms 0.9293ms 1.0761 KOps/s 1.0764 KOps/s $\color{#d91a1a}-0.03\%$
test_func_call_cm_runtime[True-compile-overhead] 0.5510ms 0.4922ms 2.0318 KOps/s 2.0132 KOps/s $\color{#35bf28}+0.92\%$
test_vmap_func_call_cm_runtime[eager] 2.8007ms 2.3098ms 432.9346 Ops/s 427.8263 Ops/s $\color{#35bf28}+1.19\%$
test_vmap_func_call_cm_runtime[compile] 0.9866ms 0.9405ms 1.0632 KOps/s 1.0558 KOps/s $\color{#35bf28}+0.70\%$
test_vmap_func_call_cm_runtime[compile-overhead] 0.5852ms 0.4949ms 2.0204 KOps/s 1.9881 KOps/s $\color{#35bf28}+1.63\%$
test_distributed 2.9499ms 0.1680ms 5.9522 KOps/s 6.4937 KOps/s $\textbf{\color{#d91a1a}-8.34\%}$
test_tdmodule 53.4210μs 27.2071μs 36.7551 KOps/s 36.6467 KOps/s $\color{#35bf28}+0.30\%$
test_tdmodule_dispatch 74.8510μs 44.3750μs 22.5352 KOps/s 22.5214 KOps/s $\color{#35bf28}+0.06\%$
test_tdseq 45.9310μs 26.3294μs 37.9804 KOps/s 37.7668 KOps/s $\color{#35bf28}+0.57\%$
test_tdseq_dispatch 68.4610μs 46.2776μs 21.6087 KOps/s 21.4951 KOps/s $\color{#35bf28}+0.53\%$
test_instantiation_functorch 2.0670ms 1.9727ms 506.9159 Ops/s 496.1299 Ops/s $\color{#35bf28}+2.17\%$
test_exec_functorch 0.2221ms 0.1718ms 5.8224 KOps/s 5.7283 KOps/s $\color{#35bf28}+1.64\%$
test_exec_functional_call 0.1879ms 0.1565ms 6.3880 KOps/s 6.4059 KOps/s $\color{#d91a1a}-0.28\%$
test_exec_td_decorator 0.4422ms 0.2300ms 4.3484 KOps/s 4.3294 KOps/s $\color{#35bf28}+0.44\%$
test_vmap_mlp_speed_decorator[True-True] 1.0079ms 0.8021ms 1.2467 KOps/s 1.2344 KOps/s $\color{#35bf28}+1.00\%$
test_vmap_mlp_speed_decorator[True-False] 0.9896ms 0.8017ms 1.2474 KOps/s 1.2296 KOps/s $\color{#35bf28}+1.44\%$
test_vmap_mlp_speed_decorator[False-True] 0.8784ms 0.6942ms 1.4405 KOps/s 1.4248 KOps/s $\color{#35bf28}+1.10\%$
test_vmap_mlp_speed_decorator[False-False] 0.8922ms 0.6965ms 1.4357 KOps/s 1.4251 KOps/s $\color{#35bf28}+0.74\%$
test_vmap_transformer_speed_decorator[True-True] 20.2178ms 20.0514ms 49.8717 Ops/s 49.5005 Ops/s $\color{#35bf28}+0.75\%$
test_vmap_transformer_speed_decorator[True-False] 20.2503ms 20.0580ms 49.8554 Ops/s 49.3897 Ops/s $\color{#35bf28}+0.94\%$
test_vmap_transformer_speed_decorator[False-True] 20.4604ms 19.8584ms 50.3565 Ops/s 50.0018 Ops/s $\color{#35bf28}+0.71\%$
test_vmap_transformer_speed_decorator[False-False] 20.4959ms 19.8819ms 50.2970 Ops/s 49.9031 Ops/s $\color{#35bf28}+0.79\%$
test_to_module_speed[True] 2.0337ms 1.4157ms 706.3645 Ops/s 701.1724 Ops/s $\color{#35bf28}+0.74\%$
test_to_module_speed[False] 1.9185ms 1.3715ms 729.1249 Ops/s 713.3270 Ops/s $\color{#35bf28}+2.21\%$
test_tc_init 79.0610μs 44.4177μs 22.5136 KOps/s 22.4001 KOps/s $\color{#35bf28}+0.51\%$
test_tc_init_tensor_only 36.6110μs 9.3191μs 107.3069 KOps/s 105.1122 KOps/s $\color{#35bf28}+2.09\%$
test_tc_init_nested 0.3543ms 87.0528μs 11.4873 KOps/s 11.3101 KOps/s $\color{#35bf28}+1.57\%$
test_tc_init_many_fields 57.9910μs 15.5867μs 64.1574 KOps/s 63.6316 KOps/s $\color{#35bf28}+0.83\%$
test_tc_first_layer_tensor 27.3500μs 1.6868μs 592.8504 KOps/s 569.7864 KOps/s $\color{#35bf28}+4.05\%$
test_tc_first_layer_tensor_only 10.0773μs 0.7066μs 1.4153 MOps/s 1.4167 MOps/s $\color{#d91a1a}-0.10\%$
test_tc_first_layer_tensor_set 41.0710μs 3.7431μs 267.1595 KOps/s 262.9466 KOps/s $\color{#35bf28}+1.60\%$
test_tc_first_layer_tensor_only_set 24.2710μs 3.2040μs 312.1118 KOps/s 328.7652 KOps/s $\textbf{\color{#d91a1a}-5.07\%}$
test_tc_first_layer_nontensor 36.5510μs 5.7480μs 173.9730 KOps/s 170.2404 KOps/s $\color{#35bf28}+2.19\%$
test_tc_second_layer_tensor 27.5410μs 4.1380μs 241.6638 KOps/s 234.5151 KOps/s $\color{#35bf28}+3.05\%$
test_tc_second_layer_nontensor 33.8310μs 8.0659μs 123.9790 KOps/s 120.3769 KOps/s $\color{#35bf28}+2.99\%$
test_unbind 0.2573s 16.8340ms 59.4035 Ops/s 72.1916 Ops/s $\textbf{\color{#d91a1a}-17.71\%}$
test_full_like 5.0369ms 4.3379ms 230.5257 Ops/s 57.3076 Ops/s $\textbf{\color{#35bf28}+302.26\%}$
test_zeros_like 10.5591ms 4.3942ms 227.5741 Ops/s 57.3414 Ops/s $\textbf{\color{#35bf28}+296.88\%}$
test_ones_like 5.0414ms 4.3892ms 227.8310 Ops/s 59.3932 Ops/s $\textbf{\color{#35bf28}+283.60\%}$
test_clone 6.8620ms 6.5712ms 152.1785 Ops/s 56.4477 Ops/s $\textbf{\color{#35bf28}+169.59\%}$
test_squeeze 66.8310μs 13.7539μs 72.7067 KOps/s 72.1681 KOps/s $\color{#35bf28}+0.75\%$
test_unsqueeze 0.1600ms 0.1084ms 9.2254 KOps/s 9.2674 KOps/s $\color{#d91a1a}-0.45\%$
test_split 0.3573ms 0.1783ms 5.6094 KOps/s 5.5142 KOps/s $\color{#35bf28}+1.73\%$
test_permute 0.2547ms 0.2042ms 4.8983 KOps/s 5.0161 KOps/s $\color{#d91a1a}-2.35\%$
test_stack 51.9664ms 51.5229ms 19.4089 Ops/s 19.4588 Ops/s $\color{#d91a1a}-0.26\%$
test_cat 51.9777ms 51.6360ms 19.3664 Ops/s 19.4512 Ops/s $\color{#d91a1a}-0.44\%$

@vmoens vmoens mentioned this pull request Feb 19, 2026
Merged
@github-actions

github-actions Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

PR Title Label Error

Unknown or invalid prefix [Benchmark].

Current title: [Benchmark] Convert plain scripts to pytest-discoverable test files

Supported Prefixes

Your PR title must start with exactly one of these prefixes:

Prefix Label Applied Example
[BugFix] BugFix [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] Documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Perf [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] Setup [Setup] Update build configuration

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

@vmoens
Update
6d82f8c 
[ghstack-poisoned]
@github-actions

github-actions Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

PR Title Label Error

Unknown or invalid prefix [Benchmark].

Current title: [Benchmark] Convert plain scripts to pytest-discoverable test files

Supported Prefixes

Your PR title must start with exactly one of these prefixes:

Prefix Label Applied Example
[BugFix] BugFix [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] Documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Perf [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] Setup [Setup] Update build configuration

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

@github-actions

github-actions Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

PR Title Label Error

Unknown or invalid prefix [Benchmark].

Current title: [Benchmark] Convert plain scripts to pytest-discoverable test files

Supported Prefixes

Your PR title must start with exactly one of these prefixes:

Prefix Label Applied Example
[BugFix] BugFix [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] Documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Perf [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] Setup [Setup] Update build configuration

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

@vmoens vmoens mentioned this pull request Feb 19, 2026
Merged
@github-actions

github-actions Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

PR Title Label Error

Unknown or invalid prefix [Benchmark].

Current title: [Benchmark] Convert plain scripts to pytest-discoverable test files

Supported Prefixes

Your PR title must start with exactly one of these prefixes:

Prefix Label Applied Example
[BugFix] BugFix [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] Documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Perf [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] Setup [Setup] Update build configuration

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

@vmoens
Update
93a93f8 
[ghstack-poisoned]
@github-actions

github-actions Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

PR Title Label Error

Unknown or invalid prefix [Benchmark].

Current title: [Benchmark] Convert plain scripts to pytest-discoverable test files

Supported Prefixes

Your PR title must start with exactly one of these prefixes:

Prefix Label Applied Example
[BugFix] BugFix [BugFix] Fix memory leak in TensorDict
[Feature] Feature [Feature] Add new storage backend
[Doc] or [Docs] Documentation [Doc] Update installation guide
[Refactor] Refactor [Refactor] Clean up module imports
[CI] CI [CI] Fix workflow permissions
[Test] or [Tests] Test [Test] Add unit tests for nn module
[Compile] Compile [Compile] Fix torch.compile issue
[Performance] or [Perf] Perf [Perf] Optimize tensor operations
[Deprecation] Deprecation [Deprecation] Mark old function
[Setup] Setup [Setup] Update build configuration

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

vmoens added a commit that referenced this pull request Feb 19, 2026
@vmoens @cursoragent
[Benchmark] Convert plain scripts to pytest-discoverable test files
9abec29 
Rename and restructure all benchmark scripts in benchmarks/ so they are
discoverable by pytest.  Each file now uses pytest.mark.skipif to skip
gracefully when infrastructure (NCCL nodes, Redis, ImageNet, etc.) is
unavailable.

- bench_distributed.py -> test_distributed_benchmarks.py
- fx_benchmarks.py -> test_fx_benchmarks.py
- storage/bench_redis.py -> storage/test_redis_benchmarks.py
- distributed/dataloading.py -> distributed/test_dataloading.py

Co-authored-by: Cursor <cursoragent@cursor.com>
ghstack-source-id: c777ff1
Pull-Request: #1594
@vmoens
vmoens deleted the branch gh/vmoens/77/base February 19, 2026 14:43
@vmoens vmoens closed this Feb 19, 2026
@vmoens
vmoens deleted the gh/vmoens/77/head branch February 19, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Benchmarks CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens