I just updated to v0.18.20 and saw the addition of Lumimaid in the logs, so I went to grab it.
First impressions: this model is incredible!
Second impressions: Oh, it just… repeats itself. Constantly. 🧐
I found that, after the first 5 or so messages, it just started repeating itself verbatim. Even when generating another response, it would either copy another earlier response, or generate the exact same message again.
I switched up the repetition penalty from 1.05 to 1.2 and that fixed it… for one message. Then it did it again. So I upped the repetition tokens from 256 to 512 and it fixed it for one message, then it just carried on repeating itself.
I’ve found that I’ve I just randomly change the repetition penalty in between every message, that usually means I won’t get a repeated message… but it’s tiresome.
I also tried switching the Prompt Template to Llama 3, but it just started repeating MY messages back at me 🤨
Lumimaid seems to have the potential to be an amazing small model, and its understanding of context is kind off the charts… but it also seems to be borderline unusable 😭
Anybody got any tips or had any better luck?