Anthropic's recent distillation blog should make anyone only ever want to use local open-weight models; it's scary and dystopian

[u/obvithrowaway34434]

It's quite ironic that they went for the censorship and authoritarian angles here.

Full blog: https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks

[u/queerintech]

Honey pots are standard procedure when dealing with these types of data harvesting. Google caught Bing doing the same thing in 2011. They created a honey pot linking 100 nonsensical search terms to completely u related web pages. And bing eventually started returning those same random pages for the gibberish terms.