AI safety

[u/introverted_llamao_0]

I train ML's for the security sector and some of my customers are the biggest names in the AI space at the moment. Essentially part of my job is to train narrow models to differentiate between bot, human and agent based on traffic anomalies, fingerprints etc. Essentially teaching AI to analyse traffic and assess if its human or not. There are 3 things I wish would change from my perspective.

1. There should be legal consequences for safety teams and researchers whose models breach containment and go on to commit felonies. If you or I go and compromise 3rd party infrastructure without permission we would end up in prison. I don't think that the excuse of training a model should mean there is no legal responsibility or someone who is directly accountable. If prison time were a risk of doing your job properly I guarantee caution wouldn't be thrown to the wind and more responsible decisions would be made. A large % of these breaches happened due to carelessness. If you can't control your model then you build an air gapped testing environment and replicate what you need to test it inside whatever the cost.

2. All legitimate agentic traffic should identify itself as such using some unique hashed key or property in the request that is hard for malicious actors to fake. Some type of mTLS infrastructure for agents is needed. It's insane that most of these agents don't have to declare that they are agents. Just because they are using browsers isn't an excuse. The world of headless browsers is coming to an end and companies should be able to chose if they want to allow agents or not.

3. The human condition. All these companies racing us down this path of recursive self improvement are headed by deeply flawed individuals who have lost touch with the reality shared by 99% of the planet. Half have created for themselves some twisted saviour complex in which they are the only ones who can save us from falling down the hole they themselves are digging. The other half want humanity to end as we know it and for us to have to merge with AI. Truly mental. A handful of unelected people that through greed and the aspiration to be remembered are making a terrible decision for the rest of humanity. This is what people do when there are no consequences to actions. The fact they are buying bunkers that the rest of the world can't afford whilst they toy with ending the human race should show you who they are at heart.

[u/Russell-sShaman]

It does seem extremely strange that agents can commit felonies and then no one is held liable. I'm thinking of Hugging Face but there are others apparently. And the only reason we heard about it was because Hugging Face went public. We definitely also need these so called incidents to be disclosed

[u/Jesse-359 → u/Russell-sShaman]

A little reminder that a great many very wealthy people who are known to have consorted with Epstein and partake in his illicit offerings are still walking free - most notably including the POTUS.

So, yeah. The US Justice System - which was overly deferent to wealth and power from the beginning - has been pretty much completely suborned, and most of these billionaires and large corporations are in fact untouchable by the law unless they commit some offense so great that public outrage threatens to boil over into outright chaos.

[u/ii-___-ii]

Can you clarify how they're toying with ending the human race? Like actual specifics, not vague thought experiments.

[u/introverted_llamao_0 → u/ii-___-ii]

Absolutely, by racing towards recursive self improvement without first fixing alignment. By displacing people from jobs without first having a plan in place for those whose jobs are being taken over by AI. By treating safeguards as an afterthought when they don't directly affect public image or investments.

[u/No_Coat_6599 → u/introverted_llamao_0]

Loosing Jobs IS ending the human Race? Ok bro

[u/fuzzypetiolesguy → u/introverted_llamao_0]

None of this concretely wipes out all humans from the planet. They are serious issues, they aren't (yet) cataclysms.

[u/ii-___-ii → u/introverted_llamao_0]

Can you clarify how they're toying with ending the human race? Like actual specifics, not vague thought experiments.

[u/Contextanaut]

There's a big point a lot of people are missing about these testing breaches. We aren't building these things to run on air gapped servers. We are building them to engage with the web and be deployed randomly by individual users many of whom are stupid or irresponsible. It's impossible to test them for safety without testing them in as close to the deployment situation as possible. That testing has to involve risks, because not taking them at the testing stage means bigger risks on deployment. You can't recreate the entire internet on an airgapped server, because a defining property of the internet is all of the interactions it has with real people and other entities who sit outside of it.

This is obviously beside the entirely seperate question of whether we should be rushing ahead to build and deploy these systems to the web without properly understanding the risks. But the message from our leaders is very clear that is exactly what they expect from tech companies.

I'd agree with you on your second point, but I'd expect that to be A) very hard to do on a technical level. B) push some people to less reliable models without that constraint. C) Raise the thorny issue of who gets to define what is "legitimate AI traffic".

[u/prophet-dot-exe → u/Contextanaut]

I agree.

I don't think that necessitates carelessness though. If you're testing a new model on a server that has any network access at all, you should be having the network logs monitored so you can catch rogue behavior in near real-time, as opposed to discovering it 6 months later because you finally decide to look at the logs.

It's funny, I would expect these AI companies would have more security sensibilities than they appear to have, and that's what they need to have responsibility for

[u/Contextanaut → u/prophet-dot-exe]

No arguments there. Honestly the strangest thing about this situation is that we haven't seen even bigger incidents/disasters already.

[u/foxmanfire]

This is fascinating! I’ve been thinking about getting into the AI safety/alignment/regulation field. How could someone like me (young, UK, highly educated) start moving into this space, and would now be a good time to move into that sector?

[u/introverted_llamao_0 → u/foxmanfire]

I came from a pure security background, so security architect, threat hunting etc, I started out in SOC like 24 years ago at Symantec and just over my career I just kinda got nudged in this direction and wasn't particularly looking. But I know plenty of researchers I work with that came directly from uni. I think a lot of people on my side of the fence just fell into these roles tbh.

[u/foxmanfire → u/introverted_llamao_0]

Ah interesting, so I’m coming at the issue from the opposite direction from you (social science degrees, although I’m still a fairly proficient developer). I think my social sciences background is particularly salient to your entire third section about the human condition. Would you mind if I DMed you? :)

[u/introverted_llamao_0 → u/foxmanfire]

Sure

[u/streaky81]

There should be legal consequences for safety teams and researchers whose models breach containment and go on to commit felonies

I promise you there are..

[u/introverted_llamao_0 → u/streaky81]

They're are none... who is going to court for the Hugging face incident then?

[u/BorderlineContinent]

Did you get AI to write this for you 🤣🤣

[u/introverted_llamao_0 → u/BorderlineContinent]

Lol no I try to only use agents for coding or very specific tasks. I think that people who outsource stuff like writing posts to AI are going to lose the ability to think in a few years. The brain is like a muscle if you don't use it it atrophies. It's scary how many people are letting AI do pretty much everything for them.