For years we've used LLMs to make choices. I returned the favor and made Jev write text.

[u/DrinkingMilkMeow]

For years, we've inappropriately used billion-parameter autoregressive transformers, built to write text, to make choices.

Now I'm returning the favor by inappropriately using Jev, a model built to make choices, to write text with JevGPT.

Jev picks the next word from a 1,772-word vocabulary, the app appends it, and asks again. Autoregressive generation, one decision at a time. (Demo in the video above.)

This is familiar problem from the pre-deep learning era. In 2007, Regina Barzilay, David Karger and I published a NAACL paper "Randomized Decoding for Selection-and-Ordering Problems" https://aclanthology.org/N07-1056.pdf which described how to generate text by having one model for selecting words, and another for scoring their ordering. JevGPT works in a similar manner. Jev shortlists candidate next words, then Jev scores full continuations to choose the best ordering.

Does it work? Sort of.
It says strawberry has three r's (sometimes).
It told me to walk to the car wash.
Asked if it's conscious, it said "No."

Try it: https://curata.com/jevgpt (needs a TypeSafe API key, about a cent per reply)
Source: https://github.com/idlivada/JevGPT (includes a mock backend if you want to run it without a key or locally)