The real potential of Llama

[BadisG]

I'm using the alpaca-7b-native-GPTQ thanks to the new fix and we can get really great outputs if you choose a good example at the begining.

Here's the character.json I'm using right now:

{
    "char_name": "### Response:",
    "char_persona": "",
    "char_greeting": "",
    "world_scenario": "Below is an instruction that describes a task. Write a response that appropriately completes the request.",
    "example_dialogue": "### Instruction: Write a poem about basketball\n{{char}}\nIn the realm of giants, where heroes play,\nA symphony of sneakers, a dance of grace,\nA tale of hardwood, of sweat and sway,\nThe orange orb takes flight,in the court's embrace.\n \nA ball, a hoop, a dream in motion,\nLeather kisses sky, to net's sweet potion,\nWith each deft dribble, a tale is spun,\nOf legends born and victories won.\n \nFrom shadows of giants, the rookies rise,\nWith hunger and ambition, they reach for the skies,\nA ballet of bodies, as they twist and leap,\nBound by the game's passion, a love so deep.\n \nThe sound of the swish, music to the ear,\nAs the ball finds the net, the crowd's roar we hear,\nCourage and spirit, in their eyes ablaze,\n In the theater of dreams, they etch their plays. \n \nWith heart and soul, they give it their all,\nEach stride, each bounce, each pass, each call,\nIn this realm of giants, where dreams unfold,\nBasketball's story is forever told."
}

I used as an example a poem written by GPT 4 and here's the result when I ask him to do another poem:

I think that a finetuning of lama with GPT 4 data would result on a really great model on it's own!

[redthista]

what model are you using, i cant find one on HF for alpaca-7b-native-GPTQ?

[BadisG]

https://huggingface.co/ozcur/alpaca-native-4bit

[Ph0rk0z]

I found alpaca-native in fp16 to be very good.. for me there was no point in using 4-bit. It stayed in character, carl actually knew what ATHF was.

The open assistant pythia based model is very good too. Similar to alpaca. These are the first 2 that impressed me to be honest.

[BadisG]

I only have 8gb of vram, I can't use the 16bit version of llama7b :(
The open assistant pythia? Is this another fine-tune of Llama?

[Ph0rk0z]

No.. it is pythia.. if you have 8gb its going to be hard.. maybe in 4bit.