Add support for reading smaller block sizes(KBs ) within a large file

[itiyama]

My application reads small random blocks within a large file. Blocks are downloaded in memory and repeated access reads the blocks directly from page cache. Today, my application uses EBS gp3 volumes, but I have been experimenting with S3 express for the same.

For small random access reads, the latency turns out be very high as even a 4KB block access results in downloading 5MB of data from express. I understand that this may be suitable for sequential access, but for random access - this results in poor latency and wasted network throughput on the instance. I assume this limitation of minimum part size as 5MB arises because of multi part download. Is there a way to turn this off?

[passaro]

Hi @itiyama, thanks for the feature request. This is something we are interested in looking at in the future.

On random access reads, the current intended behavior is for Mountpoint to make an initial GetObject request of ~1.18MB (INITIAL_READ_WINDOW_SIZE), and then only ramp up to part-sized requests (--read-part-size, which I guess you set to 5MiB) when it detects sequential reads. This is explicitly a trade-off between latency for small reads and throughput for larger sequential reads.

Would you be able to check the size of the requests triggered by the random reads in your workload? Is it 5 or ~1.18? You can see the S3 requests in Mountpoint logs, using the --debug option (see LOGGING).

Another complication: even when your application is performing a 4KB read, the kernel may request more data to populate its page cache, so Mountpoint may receive a read request of 128KiB or larger. You should be able to confirm that from the logs as well.