Getting KeyError 'max_tokens'

[madeepakkumar1]

.env

Generic

TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2
TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF
USE_MLOCK=false

Ingestion

PERSIST_DIRECTORY=db
DOCUMENTS_DIRECTORY=source_documents
INGEST_CHUNK_SIZE=500
INGEST_CHUNK_OVERLAP=50
INGEST_N_THREADS=5

Generation

MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp

MODEL_TYPE=GPT4All

MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin

MODEL_PATH=TheBloke/GPT4All-13B-snoozy-GGML/GPT4All-13B-snoozy.ggmlv3.q4_0.bin
MODEL_TEMP=0.8
MODEL_N_CTX=1024 # Max total size of prompt+answer
MODEL_MAX_TOKENS=500 # Max size of answer
MODEL_STOP=[STOP]
CHAIN_TYPE=betterstuff
N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db
N_FORWARD_DOCUMENTS=100 # How many documents to forward to the LLM, chosen among those retrieved
N_GPU_LAYERS=4

Python version

python3.10.11

System

Windows 10

CASALIOY version

latest

Information

• The official example scripts
• My own modified scripts

Related Components

• Document ingestion
• GUI
• Prompt answering

Reproduction

$python casolioy/startLLM.py

Enter a query:

Expected behavior

[madeepakkumar1]

@su77ungr Any idea how to fix it?

[madeepakkumar1]

Getting this error while loading model gptj_model_load: invalid model file 'models\TheBloke\GPT4All-13B-snoozy-GGML\GPT4Al l-13B-snoozy.ggmlv3.q4_0.bin' (bad magic

[su77ungr]

did you check the identation. theres a space in the model path at least on your comment here GPT4All-13B-snoozy.ggmlv3.q4_0.bin

[madeepakkumar1]

model get downloaded and placed at models folder

There is no space in .env

Not sure how that space is added !

[su77ungr]

Oh you are using gpt4all. so we need a gptj backend. not sure about the compatibility of that model. let me check this when im home again

[madeepakkumar1]

@su77ungr , any idea if you have checked it?