Using Kor with Transformers

[abcnow]

I checked some of your tutorials but all of them use OpenAI and I wonder if you want to use Transformers.. what would be the approach? I wonder if you can share some code for that case in specific.. thanks in advance!

[eyurtsev]

Any LLM that works with LangChain should work fine. See this tutorial: https://python.langchain.com/v0.1/docs/use_cases/extraction/

[konstantinoskalyfommatos]

@eyurtsev Could you help a bit with using LLMs from transformers to create an extraction chain?

from kor import create_extraction_chain, Object, Text

schema = Object(
id="person",
description=(
"Personal info about a person"
),
attributes=[
Text(
id="first name",
description="The first name of a person",
examples=[],
# many=True,
),

],
examples=[
    ("Alice and Bob are friends", [{'first_name': 'Alice'}, {'first_name': 'Bob'}])
]

)

llm = "?"

chain = create_extraction_chain(llm, schema)

text = "My name is Bobby and my Sister is Rachel. My brother is Joe."
output = chain.invoke(("My name is Bobby. My brother's name Joe."))
print(output)

[eyurtsev]

You can use the integration docs in langchain - https://python.langchain.com/v0.2/docs/integrations/chat/

[konstantinoskalyfommatos]

@eyurtsev Thanks for the answer, I am using the ChatOllama class with Llama3.1 and it works fine!

What if I want to use a model from transformers and it's not supported in langchain? If it helps, it's ilsp/Meltemi-7B-v1.

[eyurtsev]

Create a custom chat model: https://python.langchain.com/v0.1/docs/modules/model_io/chat/custom_chat_model/