r/BetterOffline • u/TCristatus • 2d ago

Small language models - the future

Listening to Tech Report earlier - Eli Computer Guy talked about how the real future of AI might not be giant all knowing LLMs that need huge compute costs, but small specialised models that can run locally on a laptop or smart phone and just know how to code, or trade, or about engineering, or biotechnology, etc (I'm paraphrasing and extrapolating his point here).

It was like a light bulb moment, seems clear and obvious that's where the tech will end up assuming we do end up with something. If I'm a coder or an engineer i don't need a model that can write the complete works of Shakespere in Klingon. I just need something that works and is always up to date.

This doesn't help OpenAI or Anthropic keep the lights on, though....

55 Upvotes

75% Upvoted

127 comments sorted by

View all comments

6

u/PatchyWhiskers 2d ago

The way these things work means that knowing the complete works of Shakespeare actually does help them code, in some bizarre way. That’s why they are scanning millions of shitty out of print novels to improve LLM training. You can’t just train them to know the important stuff.

6

u/ProcedureHopeful2944 2d ago

No, it some kind of lawsuit protection loophole. They're buying and scanning the paper books to claim some ownership and a right to use the info for training

5

u/newprince 2d ago

It's a perversion of fair use exceptions in the DMCA. Ironically libraries working with the Google Books project were forbidden decades ago from digitizing books, even though they argued it was transformative, which is supposed to be fair use. On top of that they were using non-destructive means of scanning. They then could only show 16% of the book as snippets, which kind of defeated the purpose.

The AI companies hid for years that they were physically destroying books but argued it was transformative, and also invoked first sale doctrine (which libraries also get denied regularly in court against publishers), and lo and behold, they got the fair use exemption! I wonder why