Parallel execution of read operations

[JimB123]

The problem/use-case that the feature addresses

Improves overall throughput and reduces average latency, better utilizing modern multi-core CPUs.

Description of the feature

Valkey guarantees the appearance of atomic isolation of command execution. Operations are applied in a strict single-threaded order, with no possibility of one command interfering with another. Valkey does NOT guarantee any ordering between clients.

The current I/O threading mechanism does the following:

• Epoll executes
• Separate threads perform the read operations in parallel
• A join occurs
• The main thread executes the individual commands sequentially

It would be possible for the IO threads to execute read operations - in parallel - BEFORE the join. We know that this is "safe" because the other main-thread operations are suspended during this period. The reads can happen in parallel, without breaking atomic isolation. After the join, the main thread would sequentially execute everything which was deemed ineligible for parallel execution.

There are some things to consider:

• Even reads perform updates to global metrics and other values
• Internally, Valkey has the concept of the "current" client - and this would be ambiguous with parallel operations
• Rehashing, expiration, LRU management, etc. could cause inadvertent modifications

One benefit of this technique is that we don't need to perform any key locking or prevent multiple access to the same key. We just ensure that reads are strictly isolated from writes.

Alternatives you've considered

I understand that Valkey is rethinking the IO threading strategy. Rather than joining the threads after the IO, the threads would (and should) continue to process in parallel with the main thread's command execution.

The above strategy could be adapted for this model. With this new model, the IO threads would be reading/parsing and essentially queuing up work for the main thread. Given this assumed queue of activities, whenever a read activity is processed, other read activities in the queue can be processed in parallel (taken out of order) as long as we can ensure that we're never taking an individual client's activities out of order.

[zuiderkwast]

Hi Jim! This was fun to read. Do you have an implementation? ;)

We could start by letting IO threads respond to PING by themselves.