[NEW] HTTP support for Valkey

[hpatro]

Subset of Valkey API access via HTTP might be handy for users to try out various features. This might be also helpful to build out
browser based Valkey cluster manager for users to administrate and manage their clusters.

[hpatro]

Few of the engine which support HTTP interface

etcd: https://etcd.io/docs/v2.3/api/
opensearch: https://opensearch.org/docs/latest/api-reference/

[zuiderkwast]

Can it be done as a module? Would we need any additional module API functions to make it possible?

[hpatro]

Can it be done as a module? Would we need any additional module API functions to make it possible?

Yeah was thinking on the same line. Trying to get a pulse of the community with the issue.

[zuiderkwast]

Another way to get HTTP could be to run Valkey from Nginx or similar.

[stockholmux]

IIRC, this was done as a module back in the Redis days in a couple of different ways (literal 1 HTTP = 1 Valkey request and then over WebSockets).

IMHO it has a lot of downsides (security, performance) but it is one of those tantalizing ideas that keeps coming up.

[hpatro]

Yeah, a regular HTTP request isn't performant enough, websockets certainly helps with the performance.
However, I'm strictly not looking it as a regular interface for application related clients, they can stick to TCP. Rather I'm thinking more in terms of providing administrator certain kind of tooling via browser that would help them avoid ssh-ing into individual nodes and reduce the operational burden of setting things up like CONFIG, ACL, also monitoring active commands, slowlog, analyzing info statistics, accessing logs, etc.

[asafpamzn]

@hpatro , do you think that it can be a client side feature?
If yes we can discuss it at https://github.com/valkey-io/valkey-glide.
How do you see this application?

[hpatro]

@hpatro , do you think that it can be a client side feature? If yes we can discuss it at https://github.com/valkey-io/valkey-glide. How do you see this application?

I don't see this interface to be a client side feature only. This protocol also needs to be supported on server side. I think a good administrator tooling experience (see the comment above) can be built via web browser on top of Valkey engine in comparison to the existing valkey-cli.

[zuiderkwast]

@asafpamzn Are you thinking of GLIDE as a proxy here?

I think GLIDE is already a protobuffer-to-RESP proxy, so it could do HTTP-to-RESP too?

[PingXie]

@asafpamzn Are you thinking of GLIDE as a proxy here?

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

[madolson]

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

Yeah, it's fairly trivial and many clients do it. The problem is that there is a lot of risk for decomposing cross-slot operations, because if you expect it to be atomic and it is not. Some clients for example break up MGET into a bunch of sub commands.

I think GLIDE is already a protobuffer-to-RESP proxy, so it could do HTTP-to-RESP too?

But why? I guess I don't really get the feature in any case, I feel like we should solidify that before moving it to glide.

[asafpamzn]

I agree with:

I feel like we should solidify that before moving it to glide.

Regarding:

slightly off topic - I have been wondering if it makes sense to emulate the standalone interface via GLIDE, i.e., a client side standalone->cluster proxy?

If you have a client that support Redis cluster why would you use the standalone interface? Unless you want to hide GLIDE behind interface of existing client. It is doable but we will find out that the devil is in the details and it is going to be a huge effort.

[hpatro]

From Dragonfly's announcement post on LI:

HTTP API! Developed directly in response to user feedback, this powerful tool is set to transform how you interact with Dragonfly, especially in the realms of edge computing and persistent storage.
Why is this feature a game-changer? Here are a few highlights:
🔗 Enhanced Connectivity: The new HTTP API allows seamless integration with your existing workflows, making it easier than ever to connect and communicate with our platform.
⚡ Optimized for Edge Computing: As more businesses move toward edge computing solutions, our HTTP API provides the flexibility and speed needed to manage data closer to where it’s generated and used. This means faster response times and improved performance for your applications.
💾 Persistent Storage Solutions: With the HTTP API, you can now efficiently manage and access your persistent storage needs, ensuring that your data is always available when you need it.

https://www.linkedin.com/posts/dragonflydb_http-api-in-dragonfly-db-a-new-feature-activity-7240058567632818176-xBuL/

[shadiramadan]

It would be great to support an HTTP interface so that you don't need to create wrappers for runtimes that don't support binary protocols. For instance- node edge runtimes cant talk to valkey/redis directly- they need to access the cache via HTTP. It would be very convenient to support this by default.