[WIP][overlap] Overlap simulation on 1d, 2d variants of llama3 for 64, 256 gpus

[IvanKobzarev]

For now requires change in pytorch to get runtime estimations for matmuls:

pytorch/pytorch#170896

python autoparallel/tools/overlap_simulator/run.py

2025-12-19 20:39:04,259 - INFO - Running overlap scheduling experiment for variant: llama3_8b_bw_256_2d_32
[rank0]:W1219 20:39:04.333000 183448 /tmp/pytorch-work/torch/_dynamo/debug_utils.py:591] no save_dir specified, will generate random data
/tmp/pytorch-work/torch/_dynamo/debug_utils.py:620: UserWarning: could not load None, generating random data instead
  warn_once(f"could not load {storage_hash}, generating random data instead")
2025-12-19 20:39:41,501 - INFO - Parsed 6 collective entries from /shared-personal/ivankobzarev/autoparallel/autoparallel/tools/overlap_simulator/colls32_8.table
2025-12-19 20:40:09,932 - INFO - Parsed 6 collective entries from /shared-personal/ivankobzarev/autoparallel/autoparallel/tools/overlap_simulator/colls32_8.table
2025-12-19 20:40:10,049 - INFO - Experiment completed successfully

Results for llama3_8b_bw_256_2d_32:
BEFORE: AG:419, RS:356, AR:65, Runtime:665.75ms
AFTER:  AG:238, RS:297, AR:65, Runtime:263.73ms
Runtime improvement: 60.39%