Improve VariableWidthBlock encoding

[pettyjamesm]

Description

Avoids converting between offsets and lengths when serializing and deserializing VariableWidthBlock instances, which enables a fast-path conversion for blocks without nulls present. When nulls are present, the compaction and expansion of offsets still outperforms the length to offset conversion.

Benchmarks

jmh.morethan.io - Significant improvements

# Before
Benchmark                                   (nullChance)  (offset)  Mode  Cnt  Score   Error  Units
BenchmarkBlockSerde.deserializeSliceDirect             0      true  avgt   10  1.139 ± 0.134  ns/op
BenchmarkBlockSerde.deserializeSliceDirect             0     false  avgt   10  1.097 ± 0.056  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .01      true  avgt   10  1.476 ± 0.026  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .01     false  avgt   10  2.066 ± 0.081  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .10      true  avgt   10  2.196 ± 0.038  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .10     false  avgt   10  2.228 ± 0.197  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .50      true  avgt   10  1.251 ± 0.013  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .50     false  avgt   10  1.234 ± 0.021  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .90      true  avgt   10  1.434 ± 0.023  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .90     false  avgt   10  1.624 ± 0.132  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .99      true  avgt   10  1.423 ± 0.069  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .99     false  avgt   10  1.443 ± 0.011  ns/op
BenchmarkBlockSerde.serializeSliceDirect               0      true  avgt   10  0.659 ± 0.010  ns/op
BenchmarkBlockSerde.serializeSliceDirect               0     false  avgt   10  0.645 ± 0.001  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .01      true  avgt   10  1.151 ± 0.039  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .01     false  avgt   10  1.162 ± 0.017  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .10      true  avgt   10  1.104 ± 0.040  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .10     false  avgt   10  1.174 ± 0.008  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .50      true  avgt   10  1.002 ± 0.014  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .50     false  avgt   10  0.962 ± 0.012  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .90      true  avgt   10  0.869 ± 0.001  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .90     false  avgt   10  0.901 ± 0.059  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .99      true  avgt   10  0.868 ± 0.012  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .99     false  avgt   10  0.796 ± 0.011  ns/op


# After
Benchmark                                   (nullChance)  (offset)  Mode  Cnt  Score   Error  Units
BenchmarkBlockSerde.deserializeSliceDirect             0      true  avgt   10  0.648 ± 0.013  ns/op
BenchmarkBlockSerde.deserializeSliceDirect             0     false  avgt   10  0.665 ± 0.013  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .01      true  avgt   10  1.302 ± 0.071  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .01     false  avgt   10  1.302 ± 0.017  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .10      true  avgt   10  1.267 ± 0.030  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .10     false  avgt   10  1.282 ± 0.074  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .50      true  avgt   10  0.973 ± 0.002  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .50     false  avgt   10  0.973 ± 0.017  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .90      true  avgt   10  0.743 ± 0.008  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .90     false  avgt   10  0.760 ± 0.058  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .99      true  avgt   10  0.688 ± 0.001  ns/op
BenchmarkBlockSerde.deserializeSliceDirect           .99     false  avgt   10  0.699 ± 0.021  ns/op
BenchmarkBlockSerde.serializeSliceDirect               0      true  avgt   10  0.538 ± 0.010  ns/op
BenchmarkBlockSerde.serializeSliceDirect               0     false  avgt   10  0.399 ± 0.033  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .01      true  avgt   10  1.035 ± 0.041  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .01     false  avgt   10  1.034 ± 0.013  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .10      true  avgt   10  1.003 ± 0.005  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .10     false  avgt   10  1.007 ± 0.004  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .50      true  avgt   10  0.909 ± 0.071  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .50     false  avgt   10  0.858 ± 0.014  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .90      true  avgt   10  0.754 ± 0.004  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .90     false  avgt   10  0.755 ± 0.001  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .99      true  avgt   10  0.725 ± 0.040  ns/op
BenchmarkBlockSerde.serializeSliceDirect             .99     false  avgt   10  0.718 ± 0.019  ns/op

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Improve performance of data exchanges involving variable width data ({issue}`27377`)

[SongChujun]

Why do we want to delete column comment?

[pettyjamesm]

The test is highly dependent on the serialized page size, so adding 4 extra bytes per serialized page breaks the test. I'm duplicating the workaround from #15760 but this is indeed a hack.

[SongChujun]

I understand, hacky part is from the test setup itself though.

[pettyjamesm]

I'm actually going to avoid changing the serialized size- we don't need to add the extra 4 bytes anyway since we can just send the non-null ending offsets and not send the starting offset of 0 in the serialized representation.

[pettyjamesm]

...and it didn't work. Reintroducing the hack.

[SongChujun]

I actually think this is not a good fix to the spurious test, since we will probably always change block serde, another code change may fail on the case where comment is deleted, but pass on the case where comment is added back. I think we should change the assertion here instead https://github.com/trinodb/trino/blob/master/testing/trino-faulttolerant-tests/src/test/java/io/trino/faulttolerant/BaseFaultTolerantExecutionTest.java#L62C20-L62C39, the two numbers just happen to be equal and is not guaranteed to be equal on all cases.

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success
Benchmark finished with status: failure
Comparing results to the static baseline values, follow above workflow link for more details/logs.
Status message: Found regressions for:<br/>(presto/tpcds, q09, totalCpuTime, over by 33.2%)
Benchmark Comparison to the closest run from Master: Report

[starburstdata-automation]

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_unpart.

Building Trino finished with status: success
Benchmark finished with status: success
Comparing results to the static baseline values, follow above workflow link for more details/logs.
Status message: NO Regression found.
Benchmark Comparison to the closest run from Master: Report