Implement Redis distributed cache without Lua

[mgravell]

Implement Redis distributed cache without Lua

Resolves #54685

This uses the same logic, but moves the logic from Lua to pipelined regular Redis commands - the same commands. This is more efficient, and avoids problems when the server is not Lua-capable.

Note that the pipeline code needs some commentary:

• for the async path it uses two overlapped async operations, observed via Task.WhenAll
• for the sync path, it uses the "batch" API in SE.Redis, which allows us to synchronously wait for multiple commands sent as though they were async, but they should be completed when it returns; hence we just check the results

Additional changes:

• stabilize tests - .NET 9 seems to have different test concurrency rules, so: added key-isolation per test
• added async expiration tests; two different code paths deserve two sets of tests

The tests are not "live" in CI, but work locally:

Build succeeded in 18.7s
Test run succeeded. Total: 98 Failed: 0 Passed: 98 Skipped: 0, Duration: 17.6s

(tests take this long because of the problems of using an external database for expiration)

[mgravell]

/asp run

[mgravell]

(rebased in a wild attempt to fix the E2E failures)

[badrishc]

I'm concerned that simply enabling and depending on client side pipelining is going to lose atomicity guarantee for multi-threaded cache servers like garnet and dragonfly. Just because the commands are pipelined doesn't guarantee they execute as a transaction in the server side. It needs to be written as multi/exec.

[mgravell]

@badrishc if you feel strongly that multi/exec is warranted: I'm fine with doing the work to do that

but ... just sayin' ... if Garnet happened to implement a custom command that was the logical equivalent of the hypothetical hsetex ... I'd totally use it if I detected Garnet :)

consider:

SOMECOMMAND key [FLAGS] seconds field value [field value ...]

that was a server-side equivalent of

HSET key field value [field value ...]
EXPIRE key seconds [FLAGS]

However, I won't hold my breath on that...

[badrishc]

@badrishc if you feel strongly that multi/exec is warranted: I'm fine with doing the work to do that

but ... just sayin' ... if Garnet happened to implement a custom command that was the logical equivalent of the hypothetical hsetex ... I'd totally use it if I detected Garnet :)

consider:

SOMECOMMAND key [FLAGS] seconds field value [field value ...]

that was a server-side equivalent of

HSET key field value [field value ...]
EXPIRE key seconds [FLAGS]

However, I won't hold my breath on that...

Per our discussion, it seems fine to go ahead with your solution. Would rather avoid multi/exec if customers aren't asking for the atomicity. Thanks.

[mgravell]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 3 pipeline(s).

[EngRajabi]

@mgravell
It is not best to implement a new implementation of IDistributedCache for Garnet instead of matching the IDistributedCache implemented for Redis with Garnet.
This has good advantages. In the future, if we want to use the special features of Garnet, our hands are open

[mgravell]

@EngRajabi we can cross that bridge trivially if we get to it; today: there would be zero differences other than "don't use Lua", which this PR harmonizes for both. We also have the ability to use server-detection in here to pick and choose features - we used to do that to choose hset vs hmset depending on the redis_version, for example - so even a single cache implementation has the capability to use implementation-specific features. Ultimately, SE.Redis is a RESP client; it doesn't actually care what the server is, as long as it can talk RESP; and both Redis and Garnet (and a bunch of others) are RESP servers.

Our hands are already: fully open.