More clarifications on the persistent repl across client

[thoriqakbar0]

hey can you elaborate on persistent REPL across the client. do you plan for api change, or were referring to something else..

I tried to look at the paper and the library and try to guess what you were trying to do and I understand it as just async multi turn. basically the multiple llm just can call one repl sometime at the same time.

I was toying around with the idea of something like:

from rlm import RLM

with RLM(persistent=True, ...) as rlm:
    r1 = rlm.completion("Q3: Revenue $50M, costs $30M. Calculate profit margin.")
    r2 = rlm.completion("Q4: Revenue $62M, costs $35M. Compare with previous quarter.")
    r3 = rlm.completion("Summarize the trend across all data.")

Is REPL state enough, or should message history to persist across calls?

tq!

[alexzhang13]

Yeah by persistence I mean: you define an RLM client with a persistent=True flag, and basically the REPLEnv will not reset after each completion call. The example you gave above is correct.

Everything else (e.g. the LLM APIs) should remain the same, it's just multiple calls with refer to the same REPL environment. We'll likely have to pass a little bit more information to the LLM if we do this (e.g. on call 2, show it what variables were made in call 1, etc.) but we can figure this out as we go.

Message history should persist across calls in the REPL environment. So they should be stored as some kind of variable in memory, but when the RLM makes a new completion, the LLM controlling it won't directly see the message history until it plays with the REPL. Hope that clears things up, and if you start a PR we can discuss this there as well!

[thoriqakbar0]

attaching my pr here: #40

also another related pr that tries to solve this is: #29

let me know what you think!. thanks!

[alexzhang13]

Will have some time tomorrow to look through them both, but I left some comments on PR #40. It looks good, but I will also need to test on my end as well.