perf: optimize the vector performance in favor of small_vector

[c8ef]

I think this PR falls under both codebase modernization and performance optimization:

1. It replaces the hand-crafted short_alloc utility with a mature, well-tested implementation from Boost (boost::container::small_vector).
2. It improves the median simple_switch_grpc throughput from 2450 Mbps to 2520 Mbps (~3% gain).

[c8ef]

Performance numbers from my laptop may not be very reliable. It is very much appreciated if someone can help me verify the result using the original mininet benchmark(simple_switch with thrift) on a more stable hardware.

Still I don't think it will block merging since the cleanup is beneficial and I'm sure it will not have poorer performance.

[fruffy]

Hmm, ideally we keep our boost footprint low. Does std::pmr work as well?

[c8ef]

Hmm, ideally we keep our boost footprint low. Does std::pmr work as well?

I think there are several reasons why small_vector is a better choice here:

1. using small_vector we can easily support move semantics.
2. pmr's allocation is a virtual call and cannot easily inlined by compiler.
3. using small_vector we can get smaller object foodprint than pmr.

[matthewtlam]

I'm okay with these changes considering that we aren't importing anything totally new and it simplifies the code. @c8ef where did you get the benchmark values from? Is there a specific benchmark test that we are using?

[c8ef]

where did you get the benchmark values from? Is there a specific benchmark test that we are using?

Mostly I am using the mininet benchmark: https://github.com/p4lang/behavioral-model/blob/main/docs/performance.md#running-the-benchmark

But I am running a modified version since building simple_switch_grpc with bazel is much easier. So it is very much appreciated if someone can help me verify the result using the original mininet benchmark(simple_switch with thrift).

[c8ef]

I also have a question about performance tuning here. After applying this patch, I ran a perf profile using the benchmark. The results indicate that the application is not compute-bound; instead, most of the time is spent in libpcap, which internally calls send/recv. While this makes sense, it's not ideal for performance optimization. Are there any general guidelines or best practices for identifying the true hotspots in this type of application?