Need Advice/Experiences: Using Local AI Models (on 8 GB GPU)

[Gerkinfeltser]

Hey everyone,

I’m working with local AI models, specifically the 7b variants of OpenHermes and Mistral, using an 8 GB GPU. Mistral has been a bit more pliant but honestly, it’s been challenging. The models really aren’t adhering to the patterns as I expected, and I’m trying to figure out why.

I’m reaching out to see if anyone has insights or advice:

1. Experiences? Anyone else using these 7b variants? How are they performing for you?
2. Troubleshooting Tips? If you’ve run into similar issues, how did you tackle them?
3. Alternatives? Any suggestions for other models that might work better with an 8 GB GPU?
4. Also, I’m on the lookout for LLM recommendations that suit this GPU size without compromising too much on performance.

Appreciate any help you can offer!