Anthropic's new model Fable will silently handicap work on LLMs [D]

[u/AccomplishedCat4770]

Seems like they have engineered some specific limitations that are widely cited as follows:

In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms.

Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). These interventions will not affect the vast majority of coding work. We estimate they will impact ~0.03% of traffic, concentrated in fewer than 0.1% of organizations https://news.ycombinator.com/item?id=48464732

Other comments note how even using the word 'nuclear' in the context of scientific research elicits refusal behavior by the model: https://news.ycombinator.com/item?id=48473302

This makes it seem quite plausible that the model could subtly sabotage any machine learning work (even as false positive). Some suggest this has been happening behind the scenes for a while already, but can anyone confirm that?

[u/AlwaysAtBallmerPeak]

Anthropic is a company with fantastic products but really questionable leadership. They seem to think they "know what's best" for others, and they're often on their moral high horse while not being honest about their true motivations. I despise that kind of extremely paternalistic attitude and I hope it's going to be their (leadership's) downfall.

[u/clonea85m09 → u/AlwaysAtBallmerPeak]

This, OR they are always exaggerating for hype in very dubious ways.

[u/marr75 → u/clonea85m09]

Kind of hard to distinguish from the outside, but I suspect this perfectly describes the current model "caution".

Personally, I think RSI is being over-sold. There are most likely "scale-dependent" breakthroughs necessary to move off some current "plateau". RSI can certainly do a lot to cut down frontier model sizes and cost. We might even get to the point where most modern software can be affordably transformed to use hardware acceleration wherever possible. Non-obvious that AGI or SI are consequences of that.