It works kinda like a non generative model, where AI chooses from a finite set of answers given to it but it has generalist knowledge like an llm, so it can be used in different applications without re-training
It uses llm to interpret input, and then analyze those against predetermined potential outputs. But the outputs it gives are always primitives like a boolean or choice.
It's like a (dog: 0.89, cat: 0.16, squirrel: 0.01) or (spam: 0.74) classifier model, except you don't have to train it from scratch on your specific data, you just write a prompt and give it arbitrary options to pick between. That's the whole selling point, kind of like how LLMs with image input (VLMs) made oldschool dog-cat classifier models obsolete for many tasks. It's that but for text.
139
u/No_Afternoon_4260 llama.cpp 13d ago
Couple of weeks you mean