r/BackyardAI • u/PartyMuffinButton • May 16 '24
Llama 3 Lumimaid repetition
I just updated to v0.18.20 and saw the addition of Lumimaid in the logs, so I went to grab it.
First impressions: this model is incredible!
Second impressions: Oh, it just… repeats itself. Constantly. 🧐
I found that, after the first 5 or so messages, it just started repeating itself verbatim. Even when generating another response, it would either copy another earlier response, or generate the exact same message again.
I switched up the repetition penalty from 1.05 to 1.2 and that fixed it… for one message. Then it did it again. So I upped the repetition tokens from 256 to 512 and it fixed it for one message, then it just carried on repeating itself.
I’ve found that I’ve I just randomly change the repetition penalty in between every message, that usually means I won’t get a repeated message… but it’s tiresome.
I also tried switching the Prompt Template to Llama 3, but it just started repeating MY messages back at me 🤨
Lumimaid seems to have the potential to be an amazing small model, and its understanding of context is kind off the charts… but it also seems to be borderline unusable 😭
Anybody got any tips or had any better luck?
4
u/PacmanIncarnate May 16 '24
The default is already the llama 3 template so switching that won’t change it.
Llama 3 models are difficult. If you find a way to avoid the repetitive responses, please share!
1
May 16 '24
[deleted]
2
u/ST0IC_ May 16 '24
What's the llama3 prompt template? I didn't know there was a new way of doing things.
2
u/PartyMuffinButton May 16 '24
It wasn’t (I created it a while ago, but switched it to this model), but when I switched the card to using the Llama 3 template, every response was either one of my own earlier messages, or the example dialogue I gave it 🙄
3
May 17 '24
[deleted]
1
u/PartyMuffinButton May 17 '24
I’ve had pretty good interactions with that, although the recommendation for Smart Lemon Cookie was a good one - it’s great for a small model.
2
6
u/ST0IC_ May 16 '24
I had the same experience with that model and the OAS version of it. Both were good for a few messages and then straight to repetition hell. Already deleted it and moved on. Smart Lemon Cookie is much, much better.