Benchmark and test IC2 performance under different workloads

[paraseba]

[dcherian]

From Seba:

One more micro benchmark that would be useful is understanding how much worse fetching a virtual chunk is compared to a native one, and if the overhead matters or not when compared with the time it takes to pull the chunk from the network.

[dcherian]

No meaningful difference between IC1 & IC2 on the benchmarks in https://github.com/earth-mover/cloud-tensor-storage-benchmarks. (thanks to third-wheel!)

[dcherian]

Python benchmarks

test_set_virtual_chunk_refs is 3X faster compared to V1 (setting 100_000 refs in 7s vs 25s)

None of the rest show meaningful differences across multiple runs (generally ~20% variance which I think is fully attributable to network/ noisy neighbour/ etc).

Here is a html report generated by Claude: report.html. I recommend scanning the boxplots first, and then scanning the ratios in the table.

Note: I didn't manage to resurrect the tigris & r2 pieces (auth issues; we should move to AL :) ) but the object_store version of s3 storage is on-par with our custom S3 storage, so I believe we are all good.

[dcherian]

With that I think this task is over. 🥳 . #1649 should be merged.