Expose standard API to allow usage from third party tools

[igor-makarov]

Currently a lot of the effort was spent on supporting different backends (OpenAI, Anthropic, etc). What I'm talking about is the front end of the stack.

I'm wondering if it could be useful to anyone to expose the RLM as a generally compatible API to allow plugging it into existing tools, such as coding agents.

I've looked at LiteLLM which has custom provider support.

@alexzhang13 thoughts?

Edit: I've hacked together an impl (not nearly prod-ready, see here: diff but it worked!

[baabakk]

Hi Igor, I have built a production API layer for RLM (FastAPI + Redis Streams queue + sync workers + SSE streaming). It is a pure overlay, and zero changes to the RLM. Happy to open a PR if there's interest here. Here's the fork: https://github.com/baabakk/rlm/tree/main.

[igor-makarov]

That's cool @baabakk however I'm not the repo's maintainer so I can't really judge this code. It does seem like a LOT - Redis, workers etc. Is this really needed? Or can the MVP be just /v1/completions doing a simple RLM call?

@alexzhang13 WDYT?

[baabakk]

RLM calls take some time, and you don't want to keep the REST connection hanging. I thought about different approaches, and this was the best way that I thought I can implement it.

[igor-makarov]

As I've said - I don't really understand much about RLMs. Thanks for doing this and let hope it's on Alex's radar.