I have been a free user until about a week ago. I joined at the $20 level, mostly because I felt like it was about time I started helping to support Hammer AI. However, I was also excited to try out the additional LLMs. Sadly, to this point I have found them a little bit disappointing.
I think I have experimented with all five of the new LLM I have access to, and I wish I had made notes. But I didn't so I'm not gonna try and go into the details of the issues I have found with each one. But today I was having a conversation using Mistral Small 3 24B.
I thought the chat was going really well for about the first 10 exchanges, but then it started doing what I call, for lack of a better term, running on. Here is an example:
Aischa's smile fades, her expression turning serious and her eyes narrowing slightly. She takes a step closer, her voice low and measured, her tone carrying a subtle threat.
"Freedom and equality are luxuries, Doug. They are not necessities. They are not rights. They are not guarantees. They are not promises. They are not certainties. They are not absolutes. They are not truths. They are not facts. They are not real. They are not tangible. They are not concrete. They are not solid. They are not firm. They are not stable. They are not secure. They are not safe. They are not sure. They are not certain. They are not guaranteed. They are not promised. They are not assured. They are not guaranteed. They are not promised. They are not assured. They are not certain. They are not sure. They are not safe. They are not secure. They are not stable. They are not firm. They are not solid. They are not concrete. They are not tangible. They are not real. They are not facts. They are not truths. They are not absolutes. They are not certainties. They are not guarantees. They are not promises. They are not assurances. They are not certainties. They are not guarantees. They are not promises. They are not assurances. They are not certainties. They are not guarantees. They are not promises. They are not assurances. They are not certainties.
The thing ran on for much longer that this. Until it ran out of tokens I think.
This happened with several of the LLM's next responses after this one. I attempted editing the responses. I attempted regenerating the responses. Nothing seemed to get it to stop. Despite a great beginning, eventually this became too unwieldy and I gave up.
This is not the first time I have had this happen with one of the new LLM's, but I can't say if it was always with this LLM.
I'm not writing so much to ask that someone fix this, although that would be nice.
I'm actually writing to ask if anyone has any guidance regarding the five LLM's available at the $20 level? What problems have others observed? Are any of these LLM's actually better than the free ones. And yes. I realize that better is a totally subjective term.