About Pipeline arg naming and chaining LLMs

[alvarobartt]

Description

As of now the Pipeline expects two args generator and labeller, but essentially those are just chained LLMs with different tasks, so the naming generator and labeller may be a bit confusing, as both LLMs may be generating something i.e. in Improving Text Embeddings with Large Language Models they generate first a set of tasks and the use those tasks combined with sampling other args to generate triplets, so at the end both LLMs are "generators" while the naming labeller for the second one can be confusing.

Additionally, we won't be able to chain more than 2 LLMs, and that could be limiting under certain scenarios, so we should carefully think on how we can provide those args so that the LLMs provided are executed sequentially, but without having to think on whether the LLM is the generator or the labeller. Also to reduce the code complexity so that the functions handling everything are common to both, making Pipeline's code more readable.

[gabrielmbmb]

Hi @alvarobartt, I agree! I think we should allow chaining LLMs in more complex ways. One simple solution could be to provide a list of LLMs that gets executed sequentially (instead of just generator and labeller), but this could be not enough for some futures pipelines.

I think the best solution is to be able to provide a DAG of LLMs, that would be flexible enough.

[sutyum]

It can get pretty difficult to get such pipelines to behave reliably (specially if using smaller models). This may be a place where an "optimisable" pipeline approach may be useful. Created an issue regarding this recently #244

[dvsrepo]

Yes, I agree too! Chaining LLMs is very important.

[davidberenstein1957]

#244