Template

[BrawnyAi]

Hi, thanks for creating oTerm. I'd like to know what I should put in the template? If I leave blank is it going to use a default template? Thanks.

[ggozad]

A pleasure :)
The template field defaults to whatever the default template is for a model. So unless you change it it should already have the default loaded.
Let me also say that the template field is there mostly for future use (for instance to make it possible to pass extra parameters to the template) and completeness. I guess that whatever customization you would do with oterm at the moment would be done by changing the system field.

Does that make sense?

[BrawnyAi]

Thanks, that's great and for clarifying the template default values!

[BrawnyAi]

Sorry to bother you again. I'm using the 7B uncensored llama2. I noticed that if the AI has to give a longer reply, it will keep repeating the system prompt. Is that because of the template or it's something I can try to change?

[ggozad]

oterm has really not much to do with what the models generate. It merely is an interface to them.
But just to give you a couple of pointers: I am assuming you haven't changed the template. If you have I would try with defaults and then figure out what causes the model to hang. But also when you go down in quantization you will experience this if you go too far.

[BrawnyAi]

I suspect it's the quantization part, something related with the number of tokens and when it uses up, it tends to add the system prompt.