Reindex implementation

[JakeDern]

Change Summary

This is a reimplementation of reindex as a part of the ongoing work of #1926. This now reindexes all the required columns, has support for metrics, and has support for dictionary encoded columns.

I also was able to uncomment most of the batching tests after this change 🥳. One more to go which requires split.

Other minor changes:

• Made it so that allowed_payload_types returns payload types in the exact same order that they are stored. This is occasionally handy to have.

Things deferred:

• Benchmarks. I had nothing to compare it to since the original didn't reindex a bunch of the necessary columns anyway like scope or resource id nor did it support dictionaries. I'll add these in when I get to the next point..
• Some optimization opportunities like using naive offsets instead of sorting and reindexing everything starting at 0. We need this path because it's possible to get into situations where we absolutely need to compact things down to fit into u16, but we can likely skip it a decent portion of the time.

What issue does this PR close?

Part of #1926.

How are these changes tested?

I added a big unit test suite.

Are there any user-facing changes?

No.

[codecov]

Codecov Report

❌ Patch coverage is 78.04878% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.62%. Comparing base (c44f94a) to head (81f17e1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2021      +/-   ##
==========================================
+ Coverage   86.40%   86.62%   +0.21%     
==========================================
  Files         526      527       +1     
  Lines      167292   168879    +1587     
==========================================
+ Hits       144551   146283    +1732     
+ Misses      22207    22062     -145     
  Partials      534      534              

Components	Coverage Δ
otap-dataflow	88.68% <78.04%> (+0.28%)	⬆️
query_abstraction	80.61% <ø> (ø)
query_engine	90.29% <ø> (ø)
syslog_cef_receivers	∅ <ø> (∅)
otel-arrow-go	53.50% <ø> (ø)
quiver	91.84% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[albertlockett]

Looks like good progress so far! I had a few small questions/comments