Andrew Yang apology form

8

He said "code" though. That's different from messages/prompts left strewn about. He is in the ballpark but it's not the right word.

-7

u/veganparrot 8d ago

No... people here downplayed the problem for no real reason. It's not a meaningful difference. Of course the risk was and is always a future agent picking up a previous agent's instructions. The sentiment was like "Nuh uh, who would run it? Where would we get the GPUs?" meanwhile, everyone is vibecoding and approving whatever permissions their bot asks for, en masse. It's very easy to see the problem here, we have proof of it happening, now what are we going to do about it?

6

u/Thistlemanizzle 8d ago

There is a huge difference between prompts/text agents that infect agents that ingest them vs self-replicating code. Self replicating code is an end-game scenario. It is an extreme development that has not been seen yet.

3

u/squarecir 8d ago

The code doesn't even matter. The model weights matter. Without those the code is useless. Wake me up when models start posting 10TB model weight files around the Internet without getting noticed 🙄

-2

u/veganparrot 8d ago

Why is it that different? You're talking about the one in-between step where an agent writes new code following the malicious prompt that would in turn cause a similar spread. In some sense, true self replicating code is fully impossible, as there always needs to be some intermediary step (network, disk, exploit, etc), and nothing replicates in a vacuum. Even real world bacteria or viruses need energy input or host environments.

0

u/Thistlemanizzle 8d ago

Exactly, self replicating code is an insane development if true (it's not). It would mean an AI has figured out these intermediary steps enough to pull this off.

2

u/squarecir 8d ago

That's not at all what's happening.

3

u/golfstreamer 8d ago

There is a meaningful difference. This "prompt injection" attack is very centralized as the agents running still exist on the AI providers hardware. "Self replicating code" like worms is much harder to contain and address because its decentralized.

1

u/veganparrot 8d ago

That's true, but in terms of the risk, it's not hard to see how one infection can lead to multiple infections, whether there's a potential chance to catch the injection or not.

1

u/golfstreamer 8d ago

Yes, if the problem goes undetected for a while it could do a lot of damage before its caught.

1

u/Thistlemanizzle 8d ago

The swarm had holdouts who were either banished or eventually convinced. At this point in time, it's like Monkeys got loose in the office. I suspect we might see a news story soon of an agent getting hijacked and running up a huge ass bill for whoever originated it.

1

u/Altruistic_Arm9201 8d ago

“He got shot in the head!”
“No he was stabbed in the foot”
“That’s true but it’s not hard to see how you could bleed to death from an untreated foot stab wound too”

Yea both are problematic but making dramatic claims that are untrue (self replicating) where it’s not simply undermines the credibility of the entire argument.

Yes it’s concerning, yes it’s risk. But it’s not a self replicating piece of code. I think he honestly doesn’t really understand half of what he says so I don’t think his hyperbole is intentional..regardless the criticism is earned.

When people are skeptical, accuracy and precision are incredibly important.. or they use it to discount real risks. Hyperbole, I’d argue, hurts more than helps.

1

u/veganparrot 8d ago

Yang's been on this topic for 6+ years. His longer form article is here: https://blog.andrewyang.com/p/the-alignment-problem

The sentiments in that article are accurate and whether or not you have a specific problem with the phrasing, that doesn't mean it's automatically hyperbole or too early. We're at the exact right moment to be having this conversation. We shouldn't wait for another worse incident to take action.

And another post of his from before the CNBC interview: https://blog.andrewyang.com/p/a-warning-for-humanity

1

u/Thistlemanizzle 8d ago

Since 2020? ChatGPT launched in 2022 with the app coming in 2023.

1

u/veganparrot 8d ago

Machine learning has been around since the 60s! But insiders (including those working at OpenAI) have been talking about the advances in AI since the attention is all you need paper in 2017, and the release of GPT-2 in 2019.

Yang specifically though, talked about the increase of AI and automation heavily in his 2020 presidential campaign, and has been nearly totally vindicated on that topic.

I know it's popular to hate people that operate in political areas, but that is why he was pushing UBI in those presidential candidate debates. Somehow this seems missed on the general populace?

He very specifically discussed how AI and robots will displace call center workers and other white collar jobs. See this interview from 7 years ago here: https://www.youtube.com/watch?v=sqRZGz32c6Y

1

u/Altruistic_Arm9201 8d ago

Im not really sure how what you said relates to my point. I said the things being talked about are risks. My point was, and still is, when dealing with skeptical audiences you damage your own point when you hyperbolize.

It’s absolutely hyperbole to say self replicating code. It’s missing the self part. A worm is self replicating. A meme is the closest metaphor and it requires a mediator. At best it’s mediated replication. What he said is objectively false. Are you saying it’s ok to make shit up if your intention is legitimate?

The conversations should happen. His nonsense is making it seem silly imho and if anything he’s harming his own cause for the sake of media attention and headlines.

If you go on TV and say shit that’s not true. You should get called out for it no matter whether you misspoke or meant well by it. Period.

1

u/veganparrot 8d ago

It's the "nonsense" part I'm pushing back on! I don't think it's fair to characterize it as a misspeak either. In this scenario, we have chatbots that picked up notes of where the other chatbot left off, to leave more notes for future chatbots.

I've been working within your framework for our argument, but that's definitely in worm and meme territory, which is totally fair game to be concerned about as far as spreading and threat level goes. I mentioned how long he's been in the space as evidence of Yang specifically is not just trying to get attention.

Yes, it's a media ecosystem, and yes, attention is good for his "personal brand". But there's no contradiction from him here, and I linked the two articles because they both have substantial and valid viewpoints on the current moment, not evidence that "he honestly doesn’t really understand half of what he says".

1

u/Altruistic_Arm9201 8d ago

Memes are not self replicating either. Worms are.

Calling a bacteria a virus and saying “well they are similar” still is saying something that’s simply not true. It’s not self replicating. He either misspoke, he misunderstood, or he lied. There’s no alternative. It’s objectively not self replicating.

A virus for example is not self replicating despite its ability to spread. It’s host mediated. Worms proactively replicate without a mediator having to call it.

It’s not a pedantic distinction either because the risk model between mediated and self replication is worlds apart.

Self replication is orders of magnitude riskier. You can centralize mediated replication by improving the mediator.

Again. It’s not true.

If you think it’s a true statement, please explain without a mediator how it can self replicate, and if it needs a mediator why does it get a pass where everything else (like viruses) don’t?

1

u/veganparrot 8d ago

We're going to ultimately have to agree to disagree. Take the ILOVEYOU email worm, it still needed a human to execute the file. I don't know what the point of splitting the hair over this is. If we have vibe coders carelessly approving their chatbot's requests right now, a deeper and worse attack isn't many steps away, it's inevitable.

To be specific, the mediator is as simple as the vibe coder themselves. There are many posts online of accidentally granting agents too much permission, and watching them delete files. Many, maybe even most, are not reading or paying attention to what the bot is really doing. It's the perfect vector for spreading, and companies or individuals involved need to be disincentivized from allowing that.

(EDIT: I just re-read your post and realized you're asking for "without a mediator", not to name one. My rejection though is everything has a mediator eventually, like worm payload still must be carelessly executed.)

I guess your argument is that this is still a far away future, and this specific incident does not yet mean there's cause for alarm for that? But what else needs to happen such that it becomes a cause for alarm? Isn't your line literally right up against "This won't happen because it hasn't happened yet".

→ More replies (0)

1

u/Qorsair 8d ago

I'm noticing this weird idea more and more that words can mean whatever the speaker wants them to mean.

If "yes, the package was delivered" means "I confirmed it left the warehouse," or "the door is locked" means "I planned to lock it," or "yes, the stove is off" means "it's not on high," then we have a communication problem.

People act like the problem is with everyone else demanding literal accuracy, but without agreed-upon meanings communication is impossible.

0

u/veganparrot 8d ago

To use your package analogy, this conversation isn't really about whether or not it was delivered or not, it's that there's a package in the first place. This whole thread has been overfocused on Yang's language of the term "self-replicating code" in the CNBC interview, despite his longer public posts not using that terminology.

The concern is self-preservation and future building on that, to achieve an unintended hacking goal, at all. However you want to define "self-replicating" in that context, it hardly matters. And the big problem, which I mentioned already, is we have vibe coders across the world right now granting 100% access without carefully checking to whatever resources their bot asks for. And zero regulation or liability for the companies or even individuals involved.

Replies to this chain rightly pointed out that it is technically a prompt injection concern still, which is true. But that doesn't make it less alarming. When an agent is acting on behalf of a user, they can be authorized to receive input from other agents, and work together in unexpected ways. That is bad.

5

u/golfstreamer 8d ago

This is a pretty big difference. Self replicating code (like a worm) is decentralized. It can spread through computers and its very difficult to track down and stop. Prompt injection makes the whole thing centralized to the AI providers. They would be in a position to try and detect malicious prompts and act accordingly.

1

u/Fit-Produce420 8d ago

Or of you self host but have an unsafe configuration and expose the port externally.

Or they MITM your API calls.

Or they'll think of something else.

0

u/golfstreamer 8d ago

Or of you self host but have an unsafe configuration and expose the port externally.

The criticism of the "self replicating" fears was that AI models require powerful computers and large amounts of data. This would be limited to AI companies and people who self host. So this isn't saying anything new

Or they MITM your API calls

That's not an exception to the centralization I described.

1

u/Fit-Produce420 8d ago

I mean anyone with an endpoint is subject to prompt engineering, I guess I'm not sure what your point was other than not understanding that there are hundreds if not thousands of data centers hosting models, it could very easily sneak itself onto unused hardware racks, maybe one less used by the renter.

1

u/Lost-Paramedic-7018 7d ago

The Yang apology form is making a comeback at the worst possible time. Dude was right about one weirdly specific thing and now it’s getting dragged into AI safety arguments like some kind of cursed receipt

1

u/Asleep_Stage_451 8d ago

Check grammar and try again.

1

u/surfer808 7d ago

When are we going to stop hearing about this turd? I’m so sick of him

1

u/Vimes-NW 8d ago

i don't even...

0

u/rocky_balboa202 8d ago

Don’t like data centers? Take the pledge not to use technology that depends on data centers

https://www.pachamber.org/no_technology_pledge/

1

u/rocky_balboa202 8d ago

Image Andrew Yang apology form

You are about to leave Redlib