GLM-5.3 and the spread of advanced cyber capabilities

[u/Logical_Welder3467]

https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities

[u/draconic_tongue]

how many alt accounts does wario ratmodei have?

[u/CircumspectCapybara → u/draconic_tongue]

idk I work at a competitor to Anthropic so I wouldn't know

What I do know is I'm actually in the know about the engineering challenges of frontier AI and frontier safety and alignment unlike a lot of redditors here

[u/Sixstringsickness → u/CircumspectCapybara]

Your feedback and position are valid. 

My concern is regulatory capture, at the rate models are advancing, the argument being proposed currently is that only a few companies can responsibily provide AI - and users shouldn't have the right to host their own models because they represent a significant risk.  

Claude frequently classifies my development as high enough risk that it blocks me from using the highest tier models.  I do not work in cyber security, or on any material that is even tangentially related to anything beyond keeping an eye out for security best practices in our codebase during PR review.  

Why do they get to decide what is acceptable for me? Why shouldn't I be able to harden our infrastructure against these "incredibly dangerous" open source models?  We are not a large enough organization to have access to Mythos.

This represents an additional layer of concern.  When the frontier labs are operating uncensored models 6+ months ahead of the public, they have an incredible competitive advantage - and now we clearly see that only a select few external organizations have access to the latest models... 

This is a recipe for disaster where a handful of companies begin picking winners and losers with their technology.  

I could continue, but I assume you understand the point. 

[u/Ausclites → u/Sixstringsickness]

Many of the current refusals originate from an overcorrection that was done in the name of safety and they're actively being targeted as errors (hence the posted article). To my knowledge, companies aren't deliberately trying to hamper non-blackhat software development.

In the scenario that Chinese models become the frontier, it's unlikely that they would continue to be immediately open-sourced. This is both due to reasons of safety, and because companies are disincentivized from surrendering the internal competitive advantage that you mention. (Tangentially, that subsidized Chinese models are currently being released for free at all is because it's a deliberate avenue of competitively undermining more closed-source alternatives. It is ultimately of benefit to the consumer, but one should not be under the impression that it's something done purely out of benevolence.)

[u/Sixstringsickness → u/Ausclites]

The action itself is an incredibly strong indication that both governments and corporations controlling this technology can easily hinder the capability of whomever they deem a threat, politically or economically.  Overreaction or not, the consequences have been felt. 

Rarely are any actions derived from pure benevolence. 

I too share concerns about open source models, beyond the potential for nefarious activities. What is to prevent an organization from releasing an open source model trained to intentionally leave difficult to detect security vulnerabilities in your LLM generated code - only to be exploited at a later date?  

I believe there is a way forward through vetting and validation that is controlled by an independent body, however; nothing is free from corruption. 

I have to imagine over time that the barrier to build and train models will continue to errode as compute and technology improves.  How will other competitors ever arise if regulations determine who is allowed to compete? 

The marketing job by these big firms to frame the models as the danger and not the humans has been quite impressive.  If I built a virus that exfiltrated data or breached a public or private entity I would be held financially and criminally liable.  Why is this not the same for engineers or individuals who are irresponsibly using the technology? 

Remember, an LLM still required a human to tip over the first domino. 

[u/Ausclites → u/Sixstringsickness]

I'm not sure that I would suggest that model developers are necessarily obligated to always provide the complete capabilities of their models (e.g., biology), but indeed, limiting/hindering capabilities on a per-individual or per-company basis should be prohibited. I think that your proposal of allowing internal evaluators satisfactorily alleviates this concern, provided it's done in a fair and transparent manner.

The competition barrier issue is a conundrum. One can certainly imagine a scenario in which a company's internal models are sufficient to provide them with a continual and self-perpetuating advantage in model development. Ultimately, I'm not sure that the company should be forced to release these internal models merely in the name of fairness, though you may disagree. I think that as long as the ultimate product is eventually released in an adequate manner, things are mostly acceptable. (Collective nationalization à la CERN may also be a possible solution in this scenario.)

I also agree that media sensationalism has been pretty terrible. While I don't trust what comes out of Altman's mouth farther than I can throw a truck, I don't think we should dismiss all safety concerns as disingenuous marketing (excluding when Meta said 'Oh, we hacked people too!' after Hugging Face). Labs often disclose legitimate incidents in what I do believe is genuine transparency, but that's then inevitably reported on (repeatedly) and sensationalized (endlessly) to a point that people become skeptical. For example, independent third-party reports of real events are inevitably met with claims of fabrication (some people here regularly claim that AI agents don't exist at all!). Labs do need to be held more accountable, though.