Any alternate local model?

[SouravaBehera]

Token indices sequence length is longer than the specified maximum sequence length for this model (2816 > 2048). Running this sequence through the model will result in indexing errors
2024-08-13 11:31:16,523 - WARNING - Prompt is too long for LLM. Chunking the input.

I am getting this warning for the mentioned model in the repo.

[Dicklesworthstone]

I would use the APIs rather than the local model. But you can also try decreasing the chunk size.

[Shahin-rmz]

Hi,
thanks for very interesting project.
I would like to use my own fine tuned local LLM (Llama3.1),
how is it possible?
Best