fix: deserialize DataColumnSidecar without cloning Uint8Array #8626

twoeths · 2025-11-22T10:19:07Z

Motivation

it's inefficient to deserialize DataColumnSidecar as we see in [fulu] performance issue deserializing DataColumnSidecar #8623, most of the time it's > 100ms due to the gc

Description

implement deserializeDataColumnSidecarUnsafe() to create a view of backed Uint8Array (Uint8Array.subarray()) instead of cloning it (Uint8Array.slice())
this is safe because all gossip message data passing through thread boundary is cloned, same to req/resp
performance is great on local benchmark but don't want to track this benchmark in CI because it's pretty clear from theory since we save a lot of memory

 deserialize DataColumnSidecar
      ✔ deserialize DataColumnSidecar with 21 blobs                         408875.0 ops/s    2.445735 us/op        -         84 runs  0.706 s
      ✔ deserializeSignedBlockHeaderUnsafe DataColumnSidecar with 21 blo 1.545595e+9 ops/s   0.6470000 ns/op        -     361190 runs  0.340 s

Closes #8623

Testing

lower gc, which will lead to lower epoch transition (confirmed that too)

job time of DataColumnSidecar is tricky, will review with team later. Need to also pull a profile to make sure

twoeths · 2025-11-22T10:21:18Z

packages/beacon-node/src/network/processor/gossipHandlers.ts

      seenTimestampSec,
    }: GossipHandlerParamGeneric<GossipType.data_column_sidecar>) => {
      const {serializedData} = gossipData;
-      const dataColumnSidecar = sszDeserialize(topic, serializedData);


if this turns out to be more efficient we need to think about upstreaming this to ssz, for example deserialize(true) for sharing Uint8Array and by default deserialize(false) to always clone it (which is what it currently is)

github-actions · 2025-11-22T11:20:31Z

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: `3ffcfdc`	Previous: `194afd5`	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	869.87 us/op	889.26 us/op	0.98
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	33.721 us/op	34.176 us/op	0.99
BLS verify - blst	815.04 us/op	1.3821 ms/op	0.59
BLS verifyMultipleSignatures 3 - blst	1.1370 ms/op	2.2657 ms/op	0.50
BLS verifyMultipleSignatures 8 - blst	1.5851 ms/op	2.5001 ms/op	0.63
BLS verifyMultipleSignatures 32 - blst	4.6362 ms/op	7.4070 ms/op	0.63
BLS verifyMultipleSignatures 64 - blst	8.6800 ms/op	10.886 ms/op	0.80
BLS verifyMultipleSignatures 128 - blst	16.585 ms/op	17.196 ms/op	0.96
BLS deserializing 10000 signatures	656.38 ms/op	683.49 ms/op	0.96
BLS deserializing 100000 signatures	6.6464 s/op	6.6704 s/op	1.00
BLS verifyMultipleSignatures - same message - 3 - blst	869.89 us/op	1.2647 ms/op	0.69
BLS verifyMultipleSignatures - same message - 8 - blst	909.07 us/op	1.6163 ms/op	0.56
BLS verifyMultipleSignatures - same message - 32 - blst	1.6153 ms/op	1.9678 ms/op	0.82
BLS verifyMultipleSignatures - same message - 64 - blst	2.4603 ms/op	2.6797 ms/op	0.92
BLS verifyMultipleSignatures - same message - 128 - blst	4.1498 ms/op	4.5102 ms/op	0.92
BLS aggregatePubkeys 32 - blst	18.784 us/op	19.633 us/op	0.96
BLS aggregatePubkeys 128 - blst	67.169 us/op	70.286 us/op	0.96
notSeenSlots=1 numMissedVotes=1 numBadVotes=10	44.385 ms/op	58.362 ms/op	0.76
notSeenSlots=1 numMissedVotes=0 numBadVotes=4	42.706 ms/op	55.582 ms/op	0.77
notSeenSlots=2 numMissedVotes=1 numBadVotes=10	33.991 ms/op	42.469 ms/op	0.80
getSlashingsAndExits - default max	70.855 us/op	70.723 us/op	1.00
getSlashingsAndExits - 2k	267.59 us/op	298.08 us/op	0.90
isKnown best case - 1 super set check	199.00 ns/op	201.00 ns/op	0.99
isKnown normal case - 2 super set checks	195.00 ns/op	200.00 ns/op	0.97
isKnown worse case - 16 super set checks	195.00 ns/op	199.00 ns/op	0.98
InMemoryCheckpointStateCache - add get delete	2.2770 us/op	2.2880 us/op	1.00
validate api signedAggregateAndProof - struct	1.3214 ms/op	1.7422 ms/op	0.76
validate gossip signedAggregateAndProof - struct	1.3421 ms/op	1.7768 ms/op	0.76
batch validate gossip attestation - vc 640000 - chunk 32	109.96 us/op	117.31 us/op	0.94
batch validate gossip attestation - vc 640000 - chunk 64	97.447 us/op	103.08 us/op	0.95
batch validate gossip attestation - vc 640000 - chunk 128	92.338 us/op	95.318 us/op	0.97
batch validate gossip attestation - vc 640000 - chunk 256	91.530 us/op	98.200 us/op	0.93
pickEth1Vote - no votes	902.06 us/op	945.41 us/op	0.95
pickEth1Vote - max votes	5.7609 ms/op	5.5466 ms/op	1.04
pickEth1Vote - Eth1Data hashTreeRoot value x2048	10.313 ms/op	10.887 ms/op	0.95
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	14.788 ms/op	20.220 ms/op	0.73
pickEth1Vote - Eth1Data fastSerialize value x2048	403.50 us/op	438.36 us/op	0.92
pickEth1Vote - Eth1Data fastSerialize tree x2048	2.6349 ms/op	2.7802 ms/op	0.95
bytes32 toHexString	334.00 ns/op	361.00 ns/op	0.93
bytes32 Buffer.toString(hex)	220.00 ns/op	244.00 ns/op	0.90
bytes32 Buffer.toString(hex) from Uint8Array	310.00 ns/op	332.00 ns/op	0.93
bytes32 Buffer.toString(hex) + 0x	223.00 ns/op	247.00 ns/op	0.90
Object access 1 prop	0.10700 ns/op	0.11700 ns/op	0.91
Map access 1 prop	0.11400 ns/op	0.12800 ns/op	0.89
Object get x1000	5.4850 ns/op	5.7390 ns/op	0.96
Map get x1000	6.0820 ns/op	6.4260 ns/op	0.95
Object set x1000	26.819 ns/op	28.063 ns/op	0.96
Map set x1000	18.552 ns/op	19.314 ns/op	0.96
Return object 10000 times	0.27110 ns/op	0.28730 ns/op	0.94
Throw Error 10000 times	3.9706 us/op	4.1768 us/op	0.95
toHex	136.62 ns/op	147.57 ns/op	0.93
Buffer.from	124.92 ns/op	130.91 ns/op	0.95
shared Buffer	80.279 ns/op	80.360 ns/op	1.00
fastMsgIdFn sha256 / 200 bytes	2.0640 us/op	2.2040 us/op	0.94
fastMsgIdFn h32 xxhash / 200 bytes	188.00 ns/op	205.00 ns/op	0.92
fastMsgIdFn h64 xxhash / 200 bytes	243.00 ns/op	271.00 ns/op	0.90
fastMsgIdFn sha256 / 1000 bytes	6.8670 us/op	7.4320 us/op	0.92
fastMsgIdFn h32 xxhash / 1000 bytes	308.00 ns/op	333.00 ns/op	0.92
fastMsgIdFn h64 xxhash / 1000 bytes	310.00 ns/op	343.00 ns/op	0.90
fastMsgIdFn sha256 / 10000 bytes	62.636 us/op	64.337 us/op	0.97
fastMsgIdFn h32 xxhash / 10000 bytes	1.7650 us/op	1.7590 us/op	1.00
fastMsgIdFn h64 xxhash / 10000 bytes	1.1810 us/op	1.1650 us/op	1.01
100 bytes - compress - snappyjs	1.2213 us/op	2.1603 us/op	0.57
100 bytes - compress - snappy	1.0258 us/op	1.1110 us/op	0.92
200 bytes - compress - snappyjs	1.6920 us/op	1.9513 us/op	0.87
200 bytes - compress - snappy	1.2286 us/op	1.2183 us/op	1.01
300 bytes - compress - snappyjs	2.1784 us/op	2.2196 us/op	0.98
300 bytes - compress - snappy	1.1603 us/op	1.2035 us/op	0.96
400 bytes - compress - snappyjs	2.4307 us/op	2.7375 us/op	0.89
400 bytes - compress - snappy	1.2215 us/op	1.2966 us/op	0.94
500 bytes - compress - snappyjs	2.6384 us/op	2.8346 us/op	0.93
500 bytes - compress - snappy	1.2426 us/op	1.3821 us/op	0.90
1000 bytes - compress - snappyjs	4.0700 us/op	5.4740 us/op	0.74
1000 bytes - compress - snappy	1.4600 us/op	1.5414 us/op	0.95
10000 bytes - compress - snappyjs	28.349 us/op	28.521 us/op	0.99
10000 bytes - compress - snappy	30.206 us/op	26.600 us/op	1.14
100 bytes - uncompress - snappyjs	746.61 ns/op	723.15 ns/op	1.03
100 bytes - uncompress - snappy	957.26 ns/op	1.0078 us/op	0.95
200 bytes - uncompress - snappyjs	1.0858 us/op	1.2312 us/op	0.88
200 bytes - uncompress - snappy	957.72 ns/op	1.0480 us/op	0.91
300 bytes - uncompress - snappyjs	1.4084 us/op	1.5380 us/op	0.92
300 bytes - uncompress - snappy	1.0004 us/op	1.3751 us/op	0.73
400 bytes - uncompress - snappyjs	1.5183 us/op	1.8571 us/op	0.82
400 bytes - uncompress - snappy	1.0963 us/op	1.1223 us/op	0.98
500 bytes - uncompress - snappyjs	1.5841 us/op	1.7190 us/op	0.92
500 bytes - uncompress - snappy	1.0745 us/op	1.1354 us/op	0.95
1000 bytes - uncompress - snappyjs	2.1729 us/op	2.1701 us/op	1.00
1000 bytes - uncompress - snappy	1.3915 us/op	1.4442 us/op	0.96
10000 bytes - uncompress - snappyjs	13.075 us/op	13.785 us/op	0.95
10000 bytes - uncompress - snappy	27.881 us/op	28.706 us/op	0.97
send data - 1000 256B messages	14.843 ms/op	15.833 ms/op	0.94
send data - 1000 512B messages	18.150 ms/op	18.713 ms/op	0.97
send data - 1000 1024B messages	25.768 ms/op	26.932 ms/op	0.96
send data - 1000 1200B messages	23.504 ms/op	25.179 ms/op	0.93
send data - 1000 2048B messages	24.631 ms/op	28.288 ms/op	0.87
send data - 1000 4096B messages	27.518 ms/op	35.682 ms/op	0.77
send data - 1000 16384B messages	43.596 ms/op	52.542 ms/op	0.83
send data - 1000 65536B messages	106.46 ms/op	110.11 ms/op	0.97
enrSubnets - fastDeserialize 64 bits	868.00 ns/op	879.00 ns/op	0.99
enrSubnets - ssz BitVector 64 bits	322.00 ns/op	346.00 ns/op	0.93
enrSubnets - fastDeserialize 4 bits	123.00 ns/op	125.00 ns/op	0.98
enrSubnets - ssz BitVector 4 bits	326.00 ns/op	344.00 ns/op	0.95
prioritizePeers score -10:0 att 32-0.1 sync 2-0	222.11 us/op	230.63 us/op	0.96
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	256.32 us/op	255.22 us/op	1.00
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	357.41 us/op	361.30 us/op	0.99
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	673.21 us/op	685.97 us/op	0.98
prioritizePeers score 0:0 att 64-1 sync 4-1	813.10 us/op	819.07 us/op	0.99
array of 16000 items push then shift	1.5443 us/op	1.5557 us/op	0.99
LinkedList of 16000 items push then shift	6.9290 ns/op	6.9960 ns/op	0.99
array of 16000 items push then pop	72.863 ns/op	75.292 ns/op	0.97
LinkedList of 16000 items push then pop	6.8810 ns/op	7.1110 ns/op	0.97
array of 24000 items push then shift	2.2885 us/op	2.3121 us/op	0.99
LinkedList of 24000 items push then shift	6.9200 ns/op	7.0070 ns/op	0.99
array of 24000 items push then pop	96.351 ns/op	98.318 ns/op	0.98
LinkedList of 24000 items push then pop	6.8360 ns/op	6.9120 ns/op	0.99
intersect bitArray bitLen 8	6.1340 ns/op	6.2320 ns/op	0.98
intersect array and set length 8	36.466 ns/op	37.888 ns/op	0.96
intersect bitArray bitLen 128	28.815 ns/op	29.208 ns/op	0.99
intersect array and set length 128	600.54 ns/op	613.03 ns/op	0.98
bitArray.getTrueBitIndexes() bitLen 128	967.00 ns/op	1.0320 us/op	0.94
bitArray.getTrueBitIndexes() bitLen 248	1.6950 us/op	1.7820 us/op	0.95
bitArray.getTrueBitIndexes() bitLen 512	3.4870 us/op	3.6610 us/op	0.95
Full columns - reconstruct all 6 blobs	68.053 us/op	66.578 us/op	1.02
Full columns - reconstruct half of the blobs out of 6	37.299 us/op	37.033 us/op	1.01
Full columns - reconstruct single blob out of 6	17.411 us/op	17.688 us/op	0.98
Half columns - reconstruct all 6 blobs	255.22 ms/op	267.50 ms/op	0.95
Half columns - reconstruct half of the blobs out of 6	128.27 ms/op	134.57 ms/op	0.95
Half columns - reconstruct single blob out of 6	46.738 ms/op	48.791 ms/op	0.96
Full columns - reconstruct all 10 blobs	122.08 us/op	135.18 us/op	0.90
Full columns - reconstruct half of the blobs out of 10	64.261 us/op	65.800 us/op	0.98
Full columns - reconstruct single blob out of 10	19.538 us/op	18.230 us/op	1.07
Half columns - reconstruct all 10 blobs	423.64 ms/op	445.58 ms/op	0.95
Half columns - reconstruct half of the blobs out of 10	216.48 ms/op	225.18 ms/op	0.96
Half columns - reconstruct single blob out of 10	46.698 ms/op	49.972 ms/op	0.93
Full columns - reconstruct all 20 blobs	241.19 us/op	239.14 us/op	1.01
Full columns - reconstruct half of the blobs out of 20	124.43 us/op	118.64 us/op	1.05
Full columns - reconstruct single blob out of 20	18.792 us/op	18.192 us/op	1.03
Half columns - reconstruct all 20 blobs	843.20 ms/op	884.15 ms/op	0.95
Half columns - reconstruct half of the blobs out of 20	422.99 ms/op	441.62 ms/op	0.96
Half columns - reconstruct single blob out of 20	46.793 ms/op	49.563 ms/op	0.94
Buffer.concat 32 items	610.00 ns/op	626.00 ns/op	0.97
Uint8Array.set 32 items	1.0820 us/op	1.9400 us/op	0.56
Buffer.copy	2.0090 us/op	2.1490 us/op	0.93
Uint8Array.set - with subarray	1.5300 us/op	1.7550 us/op	0.87
Uint8Array.set - without subarray	940.00 ns/op	917.00 ns/op	1.03
getUint32 - dataview	186.00 ns/op	207.00 ns/op	0.90
getUint32 - manual	114.00 ns/op	113.00 ns/op	1.01
Set add up to 64 items then delete first	2.1451 us/op	2.1723 us/op	0.99
OrderedSet add up to 64 items then delete first	3.1541 us/op	3.4097 us/op	0.93
Set add up to 64 items then delete last	2.3540 us/op	2.3199 us/op	1.01
OrderedSet add up to 64 items then delete last	3.4901 us/op	3.6532 us/op	0.96
Set add up to 64 items then delete middle	2.4236 us/op	2.3804 us/op	1.02
OrderedSet add up to 64 items then delete middle	5.0299 us/op	5.3441 us/op	0.94
Set add up to 128 items then delete first	4.8259 us/op	5.0048 us/op	0.96
OrderedSet add up to 128 items then delete first	8.7323 us/op	7.8131 us/op	1.12
Set add up to 128 items then delete last	4.7262 us/op	4.9351 us/op	0.96
OrderedSet add up to 128 items then delete last	7.0668 us/op	7.4887 us/op	0.94
Set add up to 128 items then delete middle	4.6202 us/op	4.8542 us/op	0.95
OrderedSet add up to 128 items then delete middle	13.617 us/op	14.644 us/op	0.93
Set add up to 256 items then delete first	9.9661 us/op	10.402 us/op	0.96
OrderedSet add up to 256 items then delete first	15.139 us/op	16.511 us/op	0.92
Set add up to 256 items then delete last	9.2296 us/op	9.6092 us/op	0.96
OrderedSet add up to 256 items then delete last	14.275 us/op	14.660 us/op	0.97
Set add up to 256 items then delete middle	9.2026 us/op	10.162 us/op	0.91
OrderedSet add up to 256 items then delete middle	40.457 us/op	43.798 us/op	0.92
transfer serialized Status (84 B)	2.1360 us/op	2.4050 us/op	0.89
copy serialized Status (84 B)	1.1720 us/op	1.2970 us/op	0.90
transfer serialized SignedVoluntaryExit (112 B)	2.2550 us/op	2.5040 us/op	0.90
copy serialized SignedVoluntaryExit (112 B)	1.2420 us/op	1.3200 us/op	0.94
transfer serialized ProposerSlashing (416 B)	2.3080 us/op	2.5530 us/op	0.90
copy serialized ProposerSlashing (416 B)	1.9960 us/op	1.4250 us/op	1.40
transfer serialized Attestation (485 B)	2.3060 us/op	2.5050 us/op	0.92
copy serialized Attestation (485 B)	1.3160 us/op	1.8350 us/op	0.72
transfer serialized AttesterSlashing (33232 B)	2.6040 us/op	2.4520 us/op	1.06
copy serialized AttesterSlashing (33232 B)	3.9660 us/op	5.1840 us/op	0.77
transfer serialized Small SignedBeaconBlock (128000 B)	4.3660 us/op	3.7070 us/op	1.18
copy serialized Small SignedBeaconBlock (128000 B)	11.635 us/op	12.298 us/op	0.95
transfer serialized Avg SignedBeaconBlock (200000 B)	4.9370 us/op	4.4900 us/op	1.10
copy serialized Avg SignedBeaconBlock (200000 B)	16.578 us/op	16.184 us/op	1.02
transfer serialized BlobsSidecar (524380 B)	4.6590 us/op	4.9120 us/op	0.95
copy serialized BlobsSidecar (524380 B)	67.445 us/op	59.483 us/op	1.13
transfer serialized Big SignedBeaconBlock (1000000 B)	5.2200 us/op	4.8280 us/op	1.08
copy serialized Big SignedBeaconBlock (1000000 B)	155.02 us/op	117.44 us/op	1.32
pass gossip attestations to forkchoice per slot	2.6407 ms/op	3.1338 ms/op	0.84
forkChoice updateHead vc 100000 bc 64 eq 0	462.90 us/op	487.07 us/op	0.95
forkChoice updateHead vc 600000 bc 64 eq 0	2.7620 ms/op	2.8852 ms/op	0.96
forkChoice updateHead vc 1000000 bc 64 eq 0	4.5979 ms/op	4.8268 ms/op	0.95
forkChoice updateHead vc 600000 bc 320 eq 0	2.7723 ms/op	2.8996 ms/op	0.96
forkChoice updateHead vc 600000 bc 1200 eq 0	2.8760 ms/op	2.9511 ms/op	0.97
forkChoice updateHead vc 600000 bc 7200 eq 0	3.0502 ms/op	3.2747 ms/op	0.93
forkChoice updateHead vc 600000 bc 64 eq 1000	2.7967 ms/op	2.9283 ms/op	0.96
forkChoice updateHead vc 600000 bc 64 eq 10000	2.9150 ms/op	3.0451 ms/op	0.96
forkChoice updateHead vc 600000 bc 64 eq 300000	9.2715 ms/op	9.5889 ms/op	0.97
computeDeltas 1400000 validators 0% inactive	13.442 ms/op	14.079 ms/op	0.95
computeDeltas 1400000 validators 10% inactive	12.860 ms/op	13.081 ms/op	0.98
computeDeltas 1400000 validators 20% inactive	11.341 ms/op	11.844 ms/op	0.96
computeDeltas 1400000 validators 50% inactive	8.6112 ms/op	8.9562 ms/op	0.96
computeDeltas 2100000 validators 0% inactive	20.394 ms/op	21.161 ms/op	0.96
computeDeltas 2100000 validators 10% inactive	18.857 ms/op	19.627 ms/op	0.96
computeDeltas 2100000 validators 20% inactive	16.920 ms/op	17.761 ms/op	0.95
computeDeltas 2100000 validators 50% inactive	12.953 ms/op	13.445 ms/op	0.96
altair processAttestation - 250000 vs - 7PWei normalcase	2.0245 ms/op	2.0644 ms/op	0.98
altair processAttestation - 250000 vs - 7PWei worstcase	2.9126 ms/op	2.9693 ms/op	0.98
altair processAttestation - setStatus - 1/6 committees join	131.51 us/op	123.31 us/op	1.07
altair processAttestation - setStatus - 1/3 committees join	252.25 us/op	242.82 us/op	1.04
altair processAttestation - setStatus - 1/2 committees join	364.51 us/op	341.01 us/op	1.07
altair processAttestation - setStatus - 2/3 committees join	460.41 us/op	441.74 us/op	1.04
altair processAttestation - setStatus - 4/5 committees join	619.73 us/op	599.09 us/op	1.03
altair processAttestation - setStatus - 100% committees join	768.31 us/op	718.13 us/op	1.07
altair processBlock - 250000 vs - 7PWei normalcase	4.3375 ms/op	4.2130 ms/op	1.03
altair processBlock - 250000 vs - 7PWei normalcase hashState	30.382 ms/op	40.098 ms/op	0.76
altair processBlock - 250000 vs - 7PWei worstcase	37.588 ms/op	42.046 ms/op	0.89
altair processBlock - 250000 vs - 7PWei worstcase hashState	69.010 ms/op	76.194 ms/op	0.91
phase0 processBlock - 250000 vs - 7PWei normalcase	1.5168 ms/op	2.4944 ms/op	0.61
phase0 processBlock - 250000 vs - 7PWei worstcase	20.756 ms/op	27.980 ms/op	0.74
altair processEth1Data - 250000 vs - 7PWei normalcase	326.49 us/op	340.44 us/op	0.96
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	6.3430 us/op	8.2800 us/op	0.77
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219	40.364 us/op	43.557 us/op	0.93
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42	9.8380 us/op	16.439 us/op	0.60
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18	6.0520 us/op	6.2240 us/op	0.97
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020	155.17 us/op	255.08 us/op	0.61
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777	1.7862 ms/op	1.8908 ms/op	0.94
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384	2.2770 ms/op	2.4295 ms/op	0.94
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384	2.3014 ms/op	2.4514 ms/op	0.94
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384	4.6287 ms/op	4.3933 ms/op	1.05
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384	2.3595 ms/op	2.4654 ms/op	0.96
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384	5.7223 ms/op	4.5677 ms/op	1.25
Tree 40 250000 create	443.75 ms/op	420.81 ms/op	1.05
Tree 40 250000 get(125000)	135.73 ns/op	140.82 ns/op	0.96
Tree 40 250000 set(125000)	1.4632 us/op	1.4590 us/op	1.00
Tree 40 250000 toArray()	16.340 ms/op	15.167 ms/op	1.08
Tree 40 250000 iterate all - toArray() + loop	15.539 ms/op	15.293 ms/op	1.02
Tree 40 250000 iterate all - get(i)	48.146 ms/op	50.363 ms/op	0.96
Array 250000 create	2.8642 ms/op	2.9282 ms/op	0.98
Array 250000 clone - spread	794.80 us/op	805.03 us/op	0.99
Array 250000 get(125000)	0.40300 ns/op	0.40400 ns/op	1.00
Array 250000 set(125000)	0.42500 ns/op	0.42700 ns/op	1.00
Array 250000 iterate all - loop	81.923 us/op	97.086 us/op	0.84
phase0 afterProcessEpoch - 250000 vs - 7PWei	40.821 ms/op	42.364 ms/op	0.96
Array.fill - length 1000000	3.4043 ms/op	3.4144 ms/op	1.00
Array push - length 1000000	13.531 ms/op	13.567 ms/op	1.00
Array.get	0.27144 ns/op	0.27953 ns/op	0.97
Uint8Array.get	0.43057 ns/op	0.44812 ns/op	0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei	15.936 ms/op	16.644 ms/op	0.96
altair processEpoch - mainnet_e81889	256.00 ms/op	297.71 ms/op	0.86
mainnet_e81889 - altair beforeProcessEpoch	16.871 ms/op	16.512 ms/op	1.02
mainnet_e81889 - altair processJustificationAndFinalization	5.3630 us/op	5.3770 us/op	1.00
mainnet_e81889 - altair processInactivityUpdates	4.0791 ms/op	4.1908 ms/op	0.97
mainnet_e81889 - altair processRewardsAndPenalties	39.737 ms/op	41.341 ms/op	0.96
mainnet_e81889 - altair processRegistryUpdates	685.00 ns/op	725.00 ns/op	0.94
mainnet_e81889 - altair processSlashings	170.00 ns/op	186.00 ns/op	0.91
mainnet_e81889 - altair processEth1DataReset	166.00 ns/op	175.00 ns/op	0.95
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.1530 ms/op	1.2162 ms/op	0.95
mainnet_e81889 - altair processSlashingsReset	845.00 ns/op	1.2120 us/op	0.70
mainnet_e81889 - altair processRandaoMixesReset	1.0820 us/op	1.2130 us/op	0.89
mainnet_e81889 - altair processHistoricalRootsUpdate	172.00 ns/op	177.00 ns/op	0.97
mainnet_e81889 - altair processParticipationFlagUpdates	494.00 ns/op	517.00 ns/op	0.96
mainnet_e81889 - altair processSyncCommitteeUpdates	135.00 ns/op	140.00 ns/op	0.96
mainnet_e81889 - altair afterProcessEpoch	50.272 ms/op	43.546 ms/op	1.15
capella processEpoch - mainnet_e217614	918.17 ms/op	1.0198 s/op	0.90
mainnet_e217614 - capella beforeProcessEpoch	60.639 ms/op	60.236 ms/op	1.01
mainnet_e217614 - capella processJustificationAndFinalization	5.1250 us/op	5.3140 us/op	0.96
mainnet_e217614 - capella processInactivityUpdates	13.911 ms/op	14.564 ms/op	0.96
mainnet_e217614 - capella processRewardsAndPenalties	182.30 ms/op	197.11 ms/op	0.92
mainnet_e217614 - capella processRegistryUpdates	6.4120 us/op	8.2980 us/op	0.77
mainnet_e217614 - capella processSlashings	172.00 ns/op	203.00 ns/op	0.85
mainnet_e217614 - capella processEth1DataReset	170.00 ns/op	178.00 ns/op	0.96
mainnet_e217614 - capella processEffectiveBalanceUpdates	4.0541 ms/op	4.2616 ms/op	0.95
mainnet_e217614 - capella processSlashingsReset	847.00 ns/op	927.00 ns/op	0.91
mainnet_e217614 - capella processRandaoMixesReset	1.0950 us/op	1.2310 us/op	0.89
mainnet_e217614 - capella processHistoricalRootsUpdate	171.00 ns/op	180.00 ns/op	0.95
mainnet_e217614 - capella processParticipationFlagUpdates	504.00 ns/op	538.00 ns/op	0.94
mainnet_e217614 - capella afterProcessEpoch	111.43 ms/op	114.27 ms/op	0.98
phase0 processEpoch - mainnet_e58758	277.70 ms/op	333.65 ms/op	0.83
mainnet_e58758 - phase0 beforeProcessEpoch	71.128 ms/op	85.089 ms/op	0.84
mainnet_e58758 - phase0 processJustificationAndFinalization	5.5920 us/op	6.1170 us/op	0.91
mainnet_e58758 - phase0 processRewardsAndPenalties	34.055 ms/op	38.057 ms/op	0.89
mainnet_e58758 - phase0 processRegistryUpdates	3.0000 us/op	3.2400 us/op	0.93
mainnet_e58758 - phase0 processSlashings	170.00 ns/op	184.00 ns/op	0.92
mainnet_e58758 - phase0 processEth1DataReset	166.00 ns/op	179.00 ns/op	0.93
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	1.1246 ms/op	1.2018 ms/op	0.94
mainnet_e58758 - phase0 processSlashingsReset	868.00 ns/op	918.00 ns/op	0.95
mainnet_e58758 - phase0 processRandaoMixesReset	1.1190 us/op	1.3560 us/op	0.83
mainnet_e58758 - phase0 processHistoricalRootsUpdate	170.00 ns/op	210.00 ns/op	0.81
mainnet_e58758 - phase0 processParticipationRecordUpdates	858.00 ns/op	983.00 ns/op	0.87
mainnet_e58758 - phase0 afterProcessEpoch	33.981 ms/op	36.168 ms/op	0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase	1.3292 ms/op	1.2718 ms/op	1.05
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	1.9306 ms/op	1.7622 ms/op	1.10
altair processInactivityUpdates - 250000 normalcase	16.942 ms/op	24.539 ms/op	0.69
altair processInactivityUpdates - 250000 worstcase	16.263 ms/op	23.998 ms/op	0.68
phase0 processRegistryUpdates - 250000 normalcase	6.4790 us/op	7.0360 us/op	0.92
phase0 processRegistryUpdates - 250000 badcase_full_deposits	222.09 us/op	448.99 us/op	0.49
phase0 processRegistryUpdates - 250000 worstcase 0.5	107.85 ms/op	132.50 ms/op	0.81
altair processRewardsAndPenalties - 250000 normalcase	26.104 ms/op	31.667 ms/op	0.82
altair processRewardsAndPenalties - 250000 worstcase	26.359 ms/op	42.874 ms/op	0.61
phase0 getAttestationDeltas - 250000 normalcase	6.7295 ms/op	16.403 ms/op	0.41
phase0 getAttestationDeltas - 250000 worstcase	6.5230 ms/op	6.6139 ms/op	0.99
phase0 processSlashings - 250000 worstcase	77.668 us/op	122.72 us/op	0.63
altair processSyncCommitteeUpdates - 250000	10.438 ms/op	11.149 ms/op	0.94
BeaconState.hashTreeRoot - No change	204.00 ns/op	230.00 ns/op	0.89
BeaconState.hashTreeRoot - 1 full validator	84.072 us/op	100.48 us/op	0.84
BeaconState.hashTreeRoot - 32 full validator	882.49 us/op	1.1530 ms/op	0.77
BeaconState.hashTreeRoot - 512 full validator	8.7696 ms/op	15.129 ms/op	0.58
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	85.982 us/op	134.12 us/op	0.64
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.6167 ms/op	2.2753 ms/op	0.71
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	20.994 ms/op	24.640 ms/op	0.85
BeaconState.hashTreeRoot - 1 balances	76.907 us/op	86.950 us/op	0.88
BeaconState.hashTreeRoot - 32 balances	849.85 us/op	1.0746 ms/op	0.79
BeaconState.hashTreeRoot - 512 balances	6.8865 ms/op	7.1659 ms/op	0.96
BeaconState.hashTreeRoot - 250000 balances	148.23 ms/op	195.79 ms/op	0.76
aggregationBits - 2048 els - zipIndexesInBitList	21.098 us/op	21.649 us/op	0.97
byteArrayEquals 32	52.477 ns/op	53.297 ns/op	0.98
Buffer.compare 32	16.680 ns/op	17.249 ns/op	0.97
byteArrayEquals 1024	1.5472 us/op	1.5703 us/op	0.99
Buffer.compare 1024	24.603 ns/op	26.555 ns/op	0.93
byteArrayEquals 16384	24.665 us/op	24.941 us/op	0.99
Buffer.compare 16384	185.10 ns/op	188.74 ns/op	0.98
byteArrayEquals 123687377	185.71 ms/op	188.77 ms/op	0.98
Buffer.compare 123687377	6.1290 ms/op	6.1236 ms/op	1.00
byteArrayEquals 32 - diff last byte	54.751 ns/op	52.104 ns/op	1.05
Buffer.compare 32 - diff last byte	16.416 ns/op	17.261 ns/op	0.95
byteArrayEquals 1024 - diff last byte	1.5220 us/op	1.5818 us/op	0.96
Buffer.compare 1024 - diff last byte	23.959 ns/op	25.745 ns/op	0.93
byteArrayEquals 16384 - diff last byte	24.307 us/op	25.090 us/op	0.97
Buffer.compare 16384 - diff last byte	187.21 ns/op	185.40 ns/op	1.01
byteArrayEquals 123687377 - diff last byte	182.55 ms/op	194.17 ms/op	0.94
Buffer.compare 123687377 - diff last byte	6.1706 ms/op	6.2372 ms/op	0.99
byteArrayEquals 32 - random bytes	4.8930 ns/op	5.2070 ns/op	0.94
Buffer.compare 32 - random bytes	16.301 ns/op	17.346 ns/op	0.94
byteArrayEquals 1024 - random bytes	4.9150 ns/op	5.1970 ns/op	0.95
Buffer.compare 1024 - random bytes	16.452 ns/op	17.320 ns/op	0.95
byteArrayEquals 16384 - random bytes	4.9540 ns/op	5.2540 ns/op	0.94
Buffer.compare 16384 - random bytes	16.556 ns/op	17.424 ns/op	0.95
byteArrayEquals 123687377 - random bytes	6.2300 ns/op	8.8000 ns/op	0.71
Buffer.compare 123687377 - random bytes	17.890 ns/op	18.520 ns/op	0.97
regular array get 100000 times	42.999 us/op	33.599 us/op	1.28
wrappedArray get 100000 times	42.833 us/op	33.578 us/op	1.28
arrayWithProxy get 100000 times	12.997 ms/op	13.892 ms/op	0.94
ssz.Root.equals	44.897 ns/op	47.063 ns/op	0.95
byteArrayEquals	44.153 ns/op	46.167 ns/op	0.96
Buffer.compare	10.084 ns/op	10.526 ns/op	0.96
processSlot - 1 slots	10.108 us/op	10.334 us/op	0.98
processSlot - 32 slots	2.4800 ms/op	3.6661 ms/op	0.68
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	2.7542 ms/op	2.9425 ms/op	0.94
getCommitteeAssignments - req 1 vs - 250000 vc	2.0253 ms/op	2.1594 ms/op	0.94
getCommitteeAssignments - req 100 vs - 250000 vc	3.9440 ms/op	4.1137 ms/op	0.96
getCommitteeAssignments - req 1000 vs - 250000 vc	4.2116 ms/op	4.3887 ms/op	0.96
findModifiedValidators - 10000 modified validators	721.77 ms/op	729.02 ms/op	0.99
findModifiedValidators - 1000 modified validators	682.72 ms/op	696.48 ms/op	0.98
findModifiedValidators - 100 modified validators	236.98 ms/op	233.52 ms/op	1.01
findModifiedValidators - 10 modified validators	125.28 ms/op	147.03 ms/op	0.85
findModifiedValidators - 1 modified validators	157.39 ms/op	151.67 ms/op	1.04
findModifiedValidators - no difference	176.34 ms/op	193.36 ms/op	0.91
compare ViewDUs	5.8627 s/op	6.2490 s/op	0.94
compare each validator Uint8Array	1.6971 s/op	1.9390 s/op	0.88
compare ViewDU to Uint8Array	1.0629 s/op	1.0515 s/op	1.01
migrate state 1000000 validators, 24 modified, 0 new	849.73 ms/op	944.91 ms/op	0.90
migrate state 1000000 validators, 1700 modified, 1000 new	1.1128 s/op	1.2683 s/op	0.88
migrate state 1000000 validators, 3400 modified, 2000 new	1.1567 s/op	1.4488 s/op	0.80
migrate state 1500000 validators, 24 modified, 0 new	784.13 ms/op	977.68 ms/op	0.80
migrate state 1500000 validators, 1700 modified, 1000 new	1.0354 s/op	1.2417 s/op	0.83
migrate state 1500000 validators, 3400 modified, 2000 new	1.2417 s/op	1.4689 s/op	0.85
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	4.1500 ns/op	4.3100 ns/op	0.96
state getBlockRootAtSlot - 250000 vs - 7PWei	644.08 ns/op	933.73 ns/op	0.69
naive computeProposerIndex 100000 validators	61.304 ms/op	57.370 ms/op	1.07
computeProposerIndex 100000 validators	1.4614 ms/op	1.5462 ms/op	0.95
naiveGetNextSyncCommitteeIndices 1000 validators	8.0828 s/op	8.3578 s/op	0.97
getNextSyncCommitteeIndices 1000 validators	120.59 ms/op	119.81 ms/op	1.01
naiveGetNextSyncCommitteeIndices 10000 validators	8.3641 s/op	8.6330 s/op	0.97
getNextSyncCommitteeIndices 10000 validators	117.15 ms/op	117.67 ms/op	1.00
naiveGetNextSyncCommitteeIndices 100000 validators	8.2774 s/op	8.3050 s/op	1.00
getNextSyncCommitteeIndices 100000 validators	116.17 ms/op	120.06 ms/op	0.97
naive computeShuffledIndex 100000 validators	25.338 s/op	26.851 s/op	0.94
cached computeShuffledIndex 100000 validators	549.92 ms/op	540.46 ms/op	1.02
naive computeShuffledIndex 2000000 validators	494.23 s/op	534.34 s/op	0.92
cached computeShuffledIndex 2000000 validators	30.006 s/op	31.649 s/op	0.95
computeProposers - vc 250000	618.48 us/op	623.92 us/op	0.99
computeEpochShuffling - vc 250000	40.856 ms/op	42.240 ms/op	0.97
getNextSyncCommittee - vc 250000	10.090 ms/op	10.629 ms/op	0.95
computeSigningRoot for AttestationData	21.985 us/op	20.909 us/op	1.05
hash AttestationData serialized data then Buffer.toString(base64)	1.5489 us/op	1.6257 us/op	0.95
toHexString serialized data	1.0500 us/op	1.1281 us/op	0.93
Buffer.toString(base64)	168.79 ns/op	162.36 ns/op	1.04
nodejs block root to RootHex using toHex	143.56 ns/op	157.02 ns/op	0.91
nodejs block root to RootHex using toRootHex	89.123 ns/op	89.287 ns/op	1.00
nodejs fromHex(blob)	110.32 us/op	114.93 us/op	0.96
nodejs fromHexInto(blob)	801.25 us/op	839.28 us/op	0.95
nodejs block root to RootHex using the deprecated toHexString	204.48 ns/op	213.02 ns/op	0.96
browser block root to RootHex using toHex	172.74 ns/op	177.72 ns/op	0.97
browser block root to RootHex using toRootHex	161.94 ns/op	168.69 ns/op	0.96
browser fromHex(blob)	770.12 us/op	800.57 us/op	0.96
browser fromHexInto(blob)	791.61 us/op	838.49 us/op	0.94
browser block root to RootHex using the deprecated toHexString	730.60 ns/op	806.70 ns/op	0.91

by benchmarkbot/action

**Motivation** - ChainSafe/lodestar#8623 - ChainSafe/lodestar#8626 **Description** - Add `reuseBytes` option which causes `deserialize` to use [`Uint8Array#subarray`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/subarray) instead of [`Uint8Array#slice`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/slice). This will cause the deserialized object to share the backing ArrayBuffer of the serialized payload. This is helpful, in some cases, to avoid additional allocations, however special care must be given to avoid corrupting memory using this option. The default behavior of `deserialize` is unchanged.

twoeths · 2025-11-26T03:19:46Z

closing in favor of #8628

codecov · 2025-11-26T03:20:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.96%. Comparing base (194afd5) to head (6c8339c).
⚠️ Report is 2 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #8626      +/-   ##
============================================
- Coverage     51.96%   51.96%   -0.01%     
============================================
  Files           848      848              
  Lines         65959    65955       -4     
  Branches       4814     4813       -1     
============================================
- Hits          34275    34272       -3     
+ Misses        31615    31614       -1     
  Partials         69       69

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

twoeths added 3 commits November 21, 2025 17:18

feat: implement deserializeDataColumnSidecarUnsafe()

68a1544

fix: apply deserializeDataColumnSidecarUnsafe() for gossip

34ae4ec

fix: apply deserializeDataColumnSidecarUnsafe() for reqresp

8ee4a18

twoeths commented Nov 22, 2025

View reviewed changes

fix: remove unused import

6c8339c

wemeetagain mentioned this pull request Nov 22, 2025

feat: add reuseBytes option to deserialize ChainSafe/ssz#503

Merged

wemeetagain mentioned this pull request Nov 22, 2025

feat: reuse bytes when deserializing gossip #8628

Open

twoeths closed this Nov 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: deserialize DataColumnSidecar without cloning Uint8Array #8626

fix: deserialize DataColumnSidecar without cloning Uint8Array #8626

Uh oh!

twoeths commented Nov 22, 2025 •

edited

Loading

Uh oh!

twoeths Nov 22, 2025

Uh oh!

github-actions bot commented Nov 22, 2025

Uh oh!

twoeths commented Nov 26, 2025

Uh oh!

codecov bot commented Nov 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

fix: deserialize DataColumnSidecar without cloning Uint8Array #8626

fix: deserialize DataColumnSidecar without cloning Uint8Array #8626

Uh oh!

Conversation

twoeths commented Nov 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

twoeths Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 22, 2025

Performance Report

Uh oh!

twoeths commented Nov 26, 2025

Uh oh!

codecov bot commented Nov 26, 2025

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

twoeths commented Nov 22, 2025 •

edited

Loading