Should we change the way we integrate models?

[rlouf]

Model integrations currently work via factory functions implemented in outlines.models:

import outlines
from transformers import LLavaModelForConditionalGeneration

model = models.openai("gpt-4o", **model_kwargs)
model = models.transformers("gpt2", model_kwargs, tokenizer_kwargs)
model = models.transformers_vision(
    "trl-internal-testing/tiny-LlavaForConditionalGeneration",
    LlavaModelForConditionalGeneration,
    model_kwargs,
    processor_kwargs,
)
model = models.llamacpp(
    "M4-ai/TinyMistral-248M-v2-Instruct-GGUF",
    "",
    **llamacpp_model_params
)

There are several issues with this interface:

1. It is not consistent.
2. It is not flexible. To pass a customer tokenizer, the user needs to import outlines.models.Transformers and pass model and tokenizers. The interface for transformers_vision is obviously ugly.
3. Outlines cannot be used as a drop-in replacement in a piece of code, we require users to re-initialize a model.
4. It needs separate interfaces for sync and async clients.

I propose the following interface instead:

import outlines

# OpenAI
from openai import OpenAI, AsyncOpenAI, AzureOpenAI

model = outlines.from_openai(OpenAI(), "gpt-4o")
model = outlines.from_openai(AsyncOpenAI(), "gpt-4o")
model = outlines.form_openai(AzureOpenAI(), "gpt-4o")

# Transformers (text and vision)
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "gpt2"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(model_name),
    AutoTokenizer.from_pretrained(model_name)
)

from transformers import AutoProcessor, LlavaForConditionalGeneration

model_name = "trl-internal-testing/tiny-LlavaForConditionalGeneration"
model = outlines.from_transformers(
    LlavaForConditionalGeneration.from_pretrained(model_name),
    AutoProcessor.from_pretrained(model_name)
)

# Llama.cpp
from llamacpp import Llama

model = outlines.from_llamacpp(Llama("model_file.gguf"))
model = outlines.from_llamacpp(Llama.from_pretrained("path", "filename"))

[ClementOLIVIER]

It's much cleaner indeed ;)

[ClementOLIVIER]

The "_from" prefix makes it more explicit, I think

[Zumus]

I like it. Much cleaner and more flexible.

[2016bgeyer]

I agree, it's cleaner

Also, slight typo (form vs from):

model = outlines.from_openai(OpenAI(), "gpt-4o")
model = outlines.from_openai(AsyncOpenAI(), "gpt-4o")
model = outlines.form_openai(AzureOpenAI(), "gpt-4o")

[shroominic]

What about
model = outlines.model("openai", "gpt-4o")
and outlines does the importing and instantiation internally?

Also allowing model = outlines.model(OpenAI(), "gpt-4o") if you want to init yourself.

And using outlines.model for all language models so you can also do
model = outlines.model("llamacpp", "model_file.gguf")

[rlouf]

We were thinking "openai:gpt-4o" down the line

[kgourgou]

I like this in particular

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "gpt2"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(model_name),
    AutoTokenizer.from_pretrained(model_name)
)

because it's flexible enough to work with models that follow the transformers API even if they don't necessarily come from that library.

[heaven00]

i don't have a lot of skin in the game here, but my 2 cents instead of outlines.from_(source), model.from_(source) might be a more explicit namespace to use.

replace source with openai, transformaers, pre_trained etc.

[Martins6]

Love it. Wouldn't change a thing. 🚀

Plus, I believe that will make it easier to integrate future models, such as litellm types of packages. You can even imagine models that have a bunch of configs to setup a workflow behind it, something like Perplexity AI.