A great proof of concept at an early stage. Keep up the good work!

[jyor0385]

I have been able to successfully run privateGPT on my windows laptop with specs:

• CPU: Intel Core i7 10th gen
• GPU: Nvidia GeForce RTX 2070
• RAM: 16Gb
• OS: Windows 10

I have only had success ingesting pdf files, though I've been told that .txt and .cvs files also work.

I have experienced performance results that are on par with the expectations given in the readme (about 30 seconds for a query).

Results from my queries have been...strange. I am using the ggmal-gpt4all-j-v1.3-groovy.bin model as recommended, but plan to try other compatible models to see if I can get better results. The responses are coming in complete sentences, but don't make a ton of sense and aren't always very accurate. But, I have been spoiled by chatGPT's abilities and probably had my expectations set a bit high.

Here is an example of a response. I have ingested four pdf documents here: a book on church constitution, a book of resources for worship services, the Westminster Confession of Faith and the ESV Bible. I asked: How many sons did Jacob have? Here is the response that I got:

The number "fourteen" is a mistranscription for an exact count or reference to fourteen specific individuals with names mentioned earlier within Genesis 29 and 30. It does not mean that Jacob had 14 total children, but rather the original text uses these numbers as placeholders when listing out those who are part of this story arc in relation to their mother's servant Bilhah (Jacob is actually a father figure).

There are pieces of accurate recall from the data ingested, but it is presented in a strange way with an even stranger focus on Bilhah. Also, according to the data ingested, Jacob had twelve sons. Not sure where the fourteen came from or how it is a "mistranscription from an exact count". Still, very impressive that it was able to reference the relevant chapters from Genesis. That itself is incredible.

This is a good step toward practical, locally hosted LLM. But it is only an early step, as the readme suggests. It is far from production ready, but I am very excited and encouraged to see this fantastic step in the right direction. Thank you to the developers for making this bold and arduous step. Keep up the good work!

[pseudotensor]

@jyor0385 Yes, both localGPT and h2oGPT (https://github.com/h2oai/h2ogpt) use better models. I tried my best to use GPT4all stuff, but it was a poor experience. Poor output or sometimes just no output, especially when adding context from vector db.

localGPT uses vicuna by default, but that's also worse than wizardlm h2oGPT uses. I recommend wizardlm uncensored. This privateGPT project should use it too instead of GPT4all IMO.