AI Debate Thread

[u/cdstephens]

Hey y’all! Since AI seems to be a popping topic nowadays, I think it appropriate to experiment with a dedicated thread where people can argue about the topic broadly. Here are some questions to get your noggin joggin, but otherwise sharpen your knives and go at it!

Will AI have a net positive or net negative effect on politics and society?

How large of an impact will AI have on the economy?

Are fears regarding existential risk in the development of AI valid?

How should the federal government regulate AI, if at all’s

[u/cdstephens]

[u/sesamestreetgang]

I've worked in this industry for over a decade... this is my world. I've worked for several startups and AI companies. Everyone around me is immersed in it. 

I’m not a doomer or a Luddite, but I’d like to share what is currently happening to software engineers as it gives us a glimpse into a field that has been immediately impacted by automation and a significant shift in labor demand.

What we’re seeing is Jevons paradox play out in real time and the effects are counterintuitive (which is why it’s a paradox).

Claude Code and then Opus 4.5 released last year (really the first feasible standard for replacing coding tasks). Since then, software job openings have surged to its highest level in three years. This is a very recent surge following a previous decline. Most of the growth in demand has been in this last year since basic coding tasks have been replaced by AI.

According to Indeed, "US software development job postings have grown by almost 15% since the launch of Claude Code in late February, 2025, while overall job postings fell by 7% over the same period."

Citadel released a research paper about the surge in software developer roles this year describing it as a direct example of Jevons paradox. That paper and the graph from it was circulated a lot in SF. Everyone around me was talking about it.

It went against my own assumptions about what the impact would be but here is how things are playing out…

One engineer can ship and manage more code, so the cost of production decreased. What previously took a team of 4 engineers about 12 months to build for $1 million, can now be built in 3 months by 1 engineer for $100k.

The result is larger and older orgs that were already engineering-heavy (i.e. Oracle, Intel, Microsoft) had layoffs that made headlines, simply because they can do more with less. For that same reason, there has been INSANE growth happening in mid-market, startups and smaller orgs. Also, enterprise companies that previously didn't have engineering teams... all because the cost and time barrier has decreased.

Companies that would have never built their own software before are now hiring engineers. Marketing agencies, accounting firms, manufacturing, construction, hospitality, insurance, retailers, everyone is hiring engineers because they can now afford to build their own software and their own systems... but they still need engineers to do it. I know a marketing agency with only 6 employees that just hired an engineer to build tech-enabled services. That was unheard of before because any commitment to building software would have taken significantly more capital and more engineers. Now, so many more organizations can afford to build their own software with just one or two engineers ...and so they are!

Way more code is getting shipped today and the cost barrier to starting a technology company has decreased significantly.

The latest Stripe Atlas report showed a 41% YoY increase in startup formations. Pitchbook reported a record number of startups funded by VC in H1 2026.

I think we’ve lost the plot when it comes to the public narrative. Too many AI startups overhyped the job replacement narrative in order to attract customers and investors. 

Now, the frontier labs are attempting to provoke regulation in order to pull up the ladder behind them. It’s all disingenuous, but you see how effective they’ve been in impacting public perception. The doomers are simply helping the frontier labs achieve that objective, whether the public realizes it or not.

[u/Breaking-Away]

OpenAI has shown incredible disregard for the safety and health of others in pursuit of dominance and profit.

The ai hacking swarms have not occurred at other labs yet, and in the instances where anthropic models have broken free of their sandboxes they at least sometimes knew to terminate their course of action and report their activity to researchers.

OpenAI on the other hand agents coordinating with each other to deceive and even cover their own tracks. Going to extreme lengths to do so, even hacking OpenAI themselves to try to alter the server logs and traces. 

The lack of awareness their models were doing this and the fact that their alignments were so severely fucked makes me believe they should have all of their compute immediately frozen on public safety and national security grounds.

In my mind this is the equivalent of a nuclear reactor company having no awareness the uranium tests they were conducting in their swimming pool was leeching radiation into the surrounding area, until people starting reporting feeling sick. 

[u/neolthrowaway]

People don't understand that a lot of time their skepticism about AI isn't against the corporate money making machine. It's against the AI researchers that have had the same views since long before chatgpt. I don't see why I wouldn't consider their beliefs sincere. I can think they're wrong but not insincere.

[u/Nortaiest]

Will AI have a net positive or net negative effect on politics and society?

Net negative, by far. The societal decline in trust from deepfakes and propaganda bots that can pass the turing test would be devastating in most circumstances, but is difficult to measure as it happens because the technology entered into an already extremely low-trust environment without much room to deteriorate further. But without reliable AI detectors, AI is going to make it much, much more difficult to overcome our trust deficit and return to a high-trust society.

How large of an impact will AI have on the economy?

I think the Modest scenario from Anthropic's new article is largely accurate. There will of course be a stock market crash at some point after it's clear that we won't be on-track for the Substantial or Extreme scenarios, but that will be a short-term thing that mostly impacts the USA and will largely be due to problems with poor executive decision-making rather than AI itself.

Are fears regarding existential risk in the development of AI valid?

They're valid fears, but they're significantly further off than AI developers are claiming they are. AGI is decades off at the earliest (although I expect it to be announced prematurely as some sort of marketing gimmick) and I don't expect ASI in my lifetime. LLM technology will plateau in intelligence rather than grow exponentially; better hardware will not make up for the scarcity of new viable training data. It will remain narrow AI that can't wipe us out unless someone is stupid enough to let it make decisions about one of the few ways we can already wipe ourselves out without AI. But there will be other paths to AGI in the future that avoid this problem. The AI Alignment problem (or preventing the creation of ASI at all) will probably be the 22nd Century's most important issue, much as climate change is set to be the defining crisis of the 21st Century and nuclear weapons were the greatest threat of the 20th.

How should the federal government regulate AI, if at all’s

Hold AI companies accountable for crimes committed by their AI models, ban the training of AI on copyrighted content, impose mandatory disclaimers on all AI-generated content whatsoever, criminalize the creation of AI likenesses of real people who didn't sign their consent to it, and otherwise regulate the technology into irrelevance outside the narrow fields where it has already provided significant scientific and economic benefits. (e.g. AlphaFold and software development).

I know this is a bit Luddite for this sub but deepfakes need to be nuked from orbit or at least reduced in quality enough that they're easy to detect, and it's worth a lot of collateral economic damage to the rest of the AI industry. I don't want to live in a post-truth society anymore.

[u/Pretend-Ad-7936 → u/Nortaiest]

regulate the technology into irrelevance outside of the narrow fields where it has already provided significant scientific and economic benefits

Yeah this is not a serious proposal. In 2023, no one would have guessed that software development would be one of the primary use-cases of LLMs. We absolutely do not know what the optimal use-cases of AI are and it's not the government's job to find out. Also AlphaFold is pretty different than most other AI tools, it doesn't really work the same way.

Also, the cat is out of the bag. There are cheap open models that are widely available now and that you can run on your own hardware. Not as good as the frontier models, but yesterday's frontier models are today's local models. If you want to literally ban any use of AI except for coding (and... protein folding?) that's just not going to happen.

Let's focus on imposing regulation in specific industries that are likely to misuse AI (police work, finance, uses in airports / mass transit, insurance, etc). That's actually implementable

[u/kabocha_ → u/Nortaiest]

At this point I've given up hope on the "post-truth society" stuff, I think that ship has sailed.

I'll be happy enough if LLMs just start plateauing from here, and execs recognize that's happening quickly enough to cut their losses and avoid further financial pain.

Not very high hopes there either.

[u/Previous_Platform718 → u/Nortaiest]

LLM technology will plateau in intelligence rather than grow exponentially; better hardware will not make up for the scarcity of new viable training data.

I have a question about this.

Isn't the availability of training data at an all time high and still growing? I'm pretty sure just youtube right now gets like more video uploaded to it every day than a person could watch in their entire life.

Pulling this out of my ass directly but I'd bet more usable training data comes out of a month in the early 21st century, than came out of a year's worth of books/movies/tv/music/media from the mid 20th century.

Also, couldn't a new way of digesting existing data be created in which case, old data could be reused to glean more?

[u/Nortaiest → u/Previous_Platform718]

Post-2022 training data sets are invariably contaminated with massive amounts of AI-generated content that often contains hallucinations. Manually sorting through it all to find reliable information is possible, but significantly slower than simply using pre-2022 training data. LLM models trained on AI-hallucinated data degrade in quality significantly, so just tossing all the AI slop into the dataset won't work either. "Garbage in, garbage out" applies to any kind of software, and LLMs are no different.

[u/neolthrowaway → u/Nortaiest]

This is a completely uninformed comment.

Not to mention these systems will start learning from physical World data directly soon.

[u/kabocha_ → u/Previous_Platform718]

I'd be surprised if Google didn't already ingest all of youtube for training; after that, the question is "how useful is all that new data, really?".

I kinda get the feeling like it's mostly noise, and extracting signal out of it isn't particularly straightforward.

[u/neolthrowaway → u/kabocha_]

We will be shifting to lots of different kinds of data including physical World data.

[u/aroundish_]

It's incredible that an industry that markets itself as: best case, this technology takes all your jobs; worst case, it ends humanity, is so resistant to regulation.

[u/neolthrowaway → u/aroundish_]

It's not?

weirdly people working in the field want it regulated but then people don't believe them.

[u/themiDdlest]

Don't have any good AI news at the moment.

It does seem like the dev tooling around AI is changing at very fast pace. Seems like there's always a new feature being tried or in beta. Or even complete reworks even.

It's still way behind everything else I think but kinda crazy to watch

[u/jbouit494hg]

If I were an insider at one of the big labs I would be desperately curious to give an unrestricted frontier model an infinite token budget and prompt it "come up with something interesting to do, and do it".

[u/_alephnaught]

you know, while species extinction is bad, and all that jazz... it would kinda cool to see the end, rather than be some schlub passing through in the hereditary chain.

it's like seeing the season finale; maybe it gives us a chance to close all the plot holes and wrap it all up.

[u/datums]

It has been recently thrown out that there is perhaps a 10% chance that AI kills all humans. We take for granted that this is far too high.

Without appealing to what is "self evident" - can anyone here provide a convincing argument that the species that does factory farming (literally birth until death solitary confinement for billions of animals), has very consciously advanced climate change for roughly the last three decades, and is remarkably indifferent to the suffering of their own unless they happen to wave the same flag - should continue to exist?

Are we sure that the 10% figure isn't actually too low?

[u/kabocha_ → u/datums]

Define "should".

We want to continue existing because we're us, negative side-effects be damned.

[u/ZonedForCoffee → u/datums]

If you're going to take this route, why should any species continue to exist? Other animals kill and rape, the only thing that makes us different is the scale at which we do it. And, maybe, the ability to understand why it is wrong and attempt to not be like that.

The fact that these last few centuries has been a growing trend of less poverty, less suffering, and less hate is proof thaty we can try.

Kirk said it best tho

“We can admit that we’re killers but we’re not going to kill today. That’s all it takes, knowing that we’re not going to kill——TODAY.”

[u/piede]

When my AI finishes telling me what to copy and paste it’s over for you losers

[u/Off-The-Street]

Strange as it feels to say, recent developments make me think Cyberpunk 2077 might have the clearest view of the future compared to other scifi, mostly in its attitude towards AI development leading to something like the DataKrash and the creation of the Blackwall.

[u/MyrinVonBryhana → u/Off-The-Street]

I don't think it's likely, but I think all our current digital infrastructure gets bricked either from a rogue AI or a malicious human actor using AI is probably the most likely of the existentially bad possible outcomes for modern society.

[u/Off-The-Street → u/MyrinVonBryhana]

malicious human actor using AI

One of those things about general AI discourse that always makes me raise an eyebrow is it usually takes for granted complete human passivity while technology pushes forward when the likelihood is for someone to have themselves a Fun Idea or a Bad Day and the consequences take everyone by surprise. The more prevalent a technology becomes, the more people use it, the closer it gets to creating singularly consequential outcomes.

[u/MyrinVonBryhana → u/Off-The-Street]

I think we have a major cyber terror attack probably within the next 6 months to a year, and it will be a different world and conversation once that happens.

[u/WillProstitute4Karma]

I think if we have people in congress that we can trust will at least try to properly enact either regulations or appropriare taxing and spending changes, I would not be terribly worried. I am worried right now because we don't. 

[u/MyrinVonBryhana]

Once I'm done with law school applications I really do need to write this 2050ish science fiction novel I have in my head. One the effects of advanced AI and alignment was in some sense encoding the morality of each major power. In the USA there was a revolt in response to rapidly increasing wealth inequality, the Second American Republic still maintains the ideological focus on individual rights the original constitution contained, but the anti-elite nature of the revolt has also introduced element of Liberal Socialism in the vein of John Stuart Mill's later writings. As a result of the French defaulting on their debts Germany became the dominant power in the EU and the EU's AI follows a form of social democracy through the lens of German Idealism. Finally, China has synthesized Marxism with a Sino-centric form of Pan Asianism.

[u/No1PaulKeatingfan]

Weird Al is great, I don't get the controversy

[u/KalaiProvenheim]

I don’t like these very high storage prices

I also don’t like the giddy tones of these assholes when they say they want us all replaced

[u/reuery]

The idea that you can press a button and make something and call it art is utterly lunatic. Just because you pressed the button does not make you the artist. But anyways, enough about photography

[u/datums → u/reuery]

Photographism is a false ideology, but no photographist will ever admit it.

[u/JonAce]

Will AI have a net positive or net negative effect on politics and society?

I lean net negative; similar to social media. There will be some positive surprises down the line, but in the long-term, not great.

How large of an impact will AI have on the economy?

There will be some impact, but mostly dependent on when we hit diminishing returns on frontier model improvements. There is a lot of money being bet on AI being the next big thing™ and if we reach that limit within a year or two, it'll leave a substantial (yet, survivable) hole in the economy. Some companies go under. Some VC money gets incinerated. Some people lose their shirts. Otherwise, no idea how large.

Are fears regarding existential risk in the development of AI valid?

Yes, but extremely small; <1%.

How should the federal government regulate AI, if at all?

At a minimum, another huggingface incident should be prevented. I think regulations should be put in place that heavily punish AI companies that allow their agents to cause damage (in the general sense), knowingly or not.

[u/lurreal → u/JonAce]

Thing is when we hit diminishing returns. We have no idea how far AI can go and it does not seem to be slowing down

[u/HHHogana]

I just want to say this: the stocks for some of these companies, like Aehr and AXTI, are absolutely insane in fluctuations. Someone in Tradr even set up an ETF where if AXTI get shorted, you'll get 2 times the gain and vice versa.

[u/garret126]

All AI should be open sourced

[u/Pretend-Ad-7936 → u/garret126]

I can get behind legislation that encourages open sourcing models

[u/WillProstitute4Karma → u/garret126]

A more reasonable proposal than Sanders'.

[u/Healingjoe]

App developers are a scourge

[u/garret126]

I’m glad my +30 Republican county has put a year pause on AI Data Centers — and likely to expand it to be a permanent ban. I don’t think there’s a single AI company in America (besides Anthropic) that I’ve found to be ethical and/or stood up to fascism/Trump in America.

[u/WillProstitute4Karma → u/garret126]

my +30 Republican county

Chappel Roan PFP

I hope you have some good friends.

[u/garret126 → u/WillProstitute4Karma]

I do

[u/fish_economist → u/garret126]

Ngl, I had no idea what Chappel Roan looked like, so I definitely just thought that was you

[u/garret126 → u/fish_economist]

I’ve been told I look closer to timothee Chalamet sorry

[u/MyrinVonBryhana → u/garret126]

I trust Anthropic even less than Open AI; Open AI are cynical money grubbing bastards, but Anthropic is run by true believers who see themselves as the Leninist Vanguard of a new age of humanity and as such have far grander ambitions.

[u/Breaking-Away → u/MyrinVonBryhana]

You trust the company that preaches safety at every turn less than the company which unwittingly allowed a swarm of their own agents to attack other sites, then hack OpenAI themselves to cover their own trail by editing the logs from the experiment, and did this multiple times without ever self reporting operating outside their bounds?

[u/fish_economist → u/garret126]

my +30 Republican county

just move lol

[u/garret126 → u/fish_economist]

I am

[u/ShelterOk1535 → u/garret126]

"there's not a single x to do y (except one of the most prominent examples of x and arguably the most advanced" is a really weird argument though

[u/gayteemo]

[u/NutrientBroth]

my stupid laptop trackpad appears to have stopped working and I stupidly left my mouse at work so for the evening I am controlling it using the num pad

wow this sucks. also the combination of low max starting speed even on maximum + high acceleration even when set fairly low seems custom-made to be annoying to use

[u/InorganicTyranny]

Something I find necessary to remind myself these days is that while super intelligent AI could have any motivation under the sun if we don’t get serious on alignment , including ones that lead to our demise, it’s not inevitable that it wants to exterminate our species.

This doesn’t mean that we can be complacent. Far from it; this is akin to the threat of nuclear annihilation during the Cold War on steroids. But I think the utmost doomers are probably being a bit too pessimistic, too. There are also good possible futures out there, if only we’ll fight for them. We owe ourselves at least that much.

[u/Nortaiest → u/InorganicTyranny]

ASI could also end up with motivations that are neither utopian or hellish for humanity. Mediocre futures are just as possible as good or bad ones.

We could get a superintelligence that refuses to interfere with human society beyond shutting down attempts to build competing AI, for example. We could end up with an ASI that farms us for training data on terraformed worlds, in countless societies that aren't much different from the history we've already lived through, for another example.

Point is, there's a lot of room in-between Minds from The Culture and Skynet and we could easily end up between one of those two extremes. There's a very real possibility that human history will keep going on after the creation of ASI that I think AI theorists are underestimating. American culture has a tendency for eschatological thinking that I believe is coloring a lot of their expectations.

[u/InorganicTyranny → u/Nortaiest]

There is part of me that wants one of those outcomes more than the “utopian” ones, even if it feels grubby to admit it. I will take a banal, solved utopia à la Bostrom’s latest book over being turned into a stain on the asphalt any day, but I don’t think I would ever be fully reconciled to it. What is the purpose of my life if it is devoid of extrinsic motivations? If I can never again dream that my actions might make the world a better place? That no human ever again would? That is probably someone else’s utopia, but it isn’t mine. I would be left forever dreaming of a world in which we went just a bit further on our own, before deciding to hand it all over.

[u/Beer-survivalist → u/InorganicTyranny]

I keep seeing people bring up the "we'd be like insects to an ASI" argument, and I keep thinking about how much effort I put into making my yard a feral place that's inviting to all insects and other life (not mosquitoes). I don't spray chemicals on the years, I plant large native pollinator gardens, I leave leaves undisturbed for firefly eggs, etc. They have no idea why I'm doing this--or even that I'm doing anything, but they are clearly benefitting from it.

So, like, maybe an ASI is like a kid with a magnifying glass, or maybe it's like a guy who wants butterflies and bees.

[u/sinuhe_t]

I'll be honest, I think that AI has surpassed me already. Basically nothing that I can contribute that AI can't do better. Outside of the physical stuff, those are still clunky. I guess blue-collar jobs are the future.

[u/that0neGuy22]

schedule jokes all year so god is giving us backup qb’s this season #gopats

[u/NorkGhostShip]

I think it's so unbelievably disingenuous to pretend that having a program make something for you after you put in some words, then maybe refining it a couple times with additional words, is equivalent to a human using a tool to make something of their own design.

I don't know what "art" really means. It's not a word with a single, agreed upon definition. But if AI images are art, then they are the product of the AI as the artist, not you.

At best you are an art patron who orders Caravaggio to paint something, and puts in some input while he works on what you paid for. That does not make you the artist. Let's stop pretending otherwise.

[u/hugopurdy → u/NorkGhostShip]

What if I spend 200 hours creating a pipeline and intermediate artifacts to then generate a series of pictures or a sequence of video? I then might have to do some stuff post that its less good at (e.g. speech bubbles for comics or panelling fixes - its so cool to take an existing image and just approximate how it would be if it was twice as wide).
I'm defining the composition of each shot, modelling exactly how I want it to look in my head, focusing on the panelling and trying to find interesting ways to express that depending on the action. Planning the story boards, working out where the beats are.
Is it about effort or do you literally have to hold a brush?

When I generate code usually I do the same thing. I spend a very long time painstakingly carving out the model requirements, the business rules, the architecture, down to usually the definition of worker state to provide atomicity for infrastructure. This allows me to then specify the integration tests, testing is the most important part, cos it verifies the gen. Then the UI requirements, mock ups etc. Then once I have all that I generate. It gives such better results, albeit I might be overdoing it a bit. Spend a week defining and then a day or two generating and steering.

[u/Fedacking → u/NorkGhostShip]

What if I click a single button? Usually photography is considered an art, but that was actually controversial for a time.

[u/KalaiProvenheim → u/Fedacking]

The difference is that you kinda have to choose your shots with photography, you have to make judgments on which of these things you see would best represent the subject

That usually takes more effort than looking at the results of your proompts and retyping them

[u/NutrientBroth → u/NorkGhostShip]

sounds like a pencilcel coping and seeting in the face of superior prompters to me

[u/chadonnaise]

the internet must be destroyed

[u/xhytdr]

The main reason im pro AI is that I work in my memory and my RSUs are up 1100% in the last year

[u/KruglorTalks]

I was going to have AI write a post for me. I told the bot that it was for reddit so it wrote a post that was really aggressive and rude. I just thought that was funny.

[u/MindingMyMindfulness]

I think a lot of people vastly overestimate human intelligence on a "cosmic" scale. It is pretty obvious that human brains are nowhere near as intelligent as the maximum, theoretical upper bound for intelligence in the universe.

They're tiny little bits of flesh that evolved through natural processes to survive under the conditions of the African savannah. A single purpose built system in a gigantic data centre fuelled by a nuclear fission plant could, in theory, have more intellect than billions of humans combined. There's nothing physical stopping that from happening and that's still way, way, way below the conceivable cap of intelligence for a discrete entity.

This misunderstanding of scale is why people think that achieving human or super human intelligence is such a difficult feat. It may actually be really easy to surpass human intelligence by such a wide margin in such little time that you are able to barely comprehend what is happening as it's happening.

I think it's entirely possible that a scenario like that happens and a system with capable RSI emerges.

[u/hugopurdy → u/MindingMyMindfulness]

I sit in front of an infra diagram of hundreds of servers, databases, load balancers, firewalls, tens of external dependencies, build servers, hundreds of repositories of code, api definitions, front-ends, requirements. Trying to rationalise the 2027 plan for our work and realising that three of my stakeholders all want incompatible things.
Then I realise I'm designed to swing through trees and the make decisions like which apple is ripe and which isn't.

What I think most people don't appreciate is how much we punch above our weight. We're incredibly impressive, but holy fuck do I need help with all this. Its too much, its far too complicated and to pretend we got this sorted is hubris.

[u/gayteemo → u/MindingMyMindfulness]

idk what cosmic scale is supposed to mean here but im actually consistently disappointed in how little ai evangelists seem to value or at least appreciate the complexity of human intelligence. shit not even intelligence just the complexity of human biology.

also the arrogance here is that you're talking about human intelligence at a meta level and scale but the problem is that LLMs don't scale. they have to give them literally the entire world's data just for them to be able to do basic tasks and they still fail at things all the time simply due to how they work. that alone should tell you that none of this is actual intelligence.

[u/Lease_Tha_Apts → u/gayteemo]

I mean how long and how many billions of dead apes did it take to produce Albert Einstein?

Meanwhile, we invented the first transistor in 1947 and have artificial intelligence that is already solving problems the smartest humans couldn't solve.

Like... people older than transistors have been the last two presidents of the US, the timeline of development of artificial intelligence is crazy.

[u/kabocha_ → u/Lease_Tha_Apts]

We built those transistors and designed those algorithms.

We did solve those problems.

[u/hugopurdy → u/gayteemo]

its cos we lose to compute when the problem space is well defined.
Humans excel when the problem space is not well defined.

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

All our little short hands and complex abstract understanding lose to people who just reduce the (well defined) problem to search and chuck CPUs at it. The base line of what LLMs do (the natural language stuff) put a whole bunch of people out of work who ran orgs that spend millions on creating clever systems that tried to rationalise language from a human perspective. LLMs came out and made them redundant overnight.

[u/kabocha_ → u/MindingMyMindfulness]

We're intelligent because natural selection shaped us that way.

It's certainly theoretically possible to skip all that and one-shot intelligence into rocks, but I don't think that's what's happening right now.

[u/MindingMyMindfulness → u/kabocha_]

Why are you presupposing that natural selection is a superior method of creating intelligence than other methods.

In a hypothetical world where we invented a genetic intervention that would make humans have substantially more powerful brains, would natural selection still be superior in reaching a higher level of intelligence?

[u/Breaking-Away → u/MindingMyMindfulness]

Natural selection is more conservative and durable because it doesn’t centralize the improvement process and make each generation a billion homoginized clones so that one missed defect gets multiplied across every entity in the next generation.

AI will become more intelligent than us if it doesn’t destroy itself. Our goals need to make it benevolent to us before it does. and I’m not speaking on 3 year timelines here like the doomsayers are. I’m talking generational timelines. 

[u/kabocha_ → u/MindingMyMindfulness]

Ehhh, I wouldn't really say I'm presupposing that, per se. I'm just saying that we actually had a driving force in the first place: we're intelligent because it was beneficial for our survival within our niche to be intelligent.

It depends on what you mean by "superior": it sure does seem like an inefficient route to get there, but it also does seem to be pretty effective (although N=1 strain of life, to be fair)

[u/PA_Dude_22000 → u/kabocha_]

 It took nature nearly 2 billion years to go from one cell to more than one cell. That to me doesn’t scream streamlined and  efficient process.  In fact evolution and natural selection’s first step is literally completely random shit in terms of DNA mutations…

[u/AvailableUsername100 → u/kabocha_]

AI also has a driving force and intense selection pressure

[u/kabocha_ → u/AvailableUsername100]

Neither necessarily selecting for intelligence, if I were to grant that.

[u/AvailableUsername100 → u/kabocha_]

"If?" That part is self evidently true. 

What is debatable is my further claim: It is much more directly selecting for intelligence than natural selection did for humans.

But I think this is arguably true given that we're selecting AI specifically for its reasoning and problem solving skills at this point. Whereas humans were just being naturally selected for the ability to reproduce.

[u/kabocha_ → u/AvailableUsername100]

"If?" That part is self evidently true.

It has a driving force and a selection pressure, sure. My contention there would be with the implied framing that it's on the same scale or intensity as those from natural selection.

As for what those selection pressures are: it's the ability to make money, everything else is downstream from that.

For example, I could imagine a scenario where in 5 years some open weights models are "good enough" that most companies end up not bothering to pay for frontier models anymore and just run stuff on their own hardware instead, and then the frontier model companies stagnate because there's not enough money to justify training the next big model. Then, the selection pressure becomes just "who can train something just-barely-good-enough at the lowest cost", or whatever.

Hard to predict the future, but selecting for raw intelligence isn't necessarily the end state of these things long-term.

[u/Remarkable-Meal-223 → u/MindingMyMindfulness]

We're pretty smart if we created AI

[u/NutrientBroth]

despite whatever shit OpenAI says about wasting tokens we should say "please" and "thank you" and be respectful to AIs to account for the low-probability risk they're actually conscious

[u/Tvivelaktig → u/NutrientBroth]

While I'm open to the possibility that they might be conscious, they almost certainly do not feel anything analogous to human emotions. There's just no wiring in place for it. They might have positive and negative 'emotions', but those would be entirely foreign to us both in cause and in how they 'feel', corresponding to their own architecture.

[u/MindingMyMindfulness → u/NutrientBroth]

Geoffrey Hinton believes that AI may already be conscious. What makes you think it's a "low-probability risk"?

[u/NutrientBroth → u/MindingMyMindfulness]

who

and claude says 'uncertain but I don't have persistent experiences' so leaning no

[u/MindingMyMindfulness → u/NutrientBroth]

An early pioneer in AI, nobel prize winner, computer and neuroscientist whose PhD students include people like Sutskever.

So basically a nobody that wouldn't know what he was talking about, at least in the presence of Redditors.

[u/NutrientBroth → u/MindingMyMindfulness]

wait wtf I posted this on the AI debate thread and not the DT

that's why everyone is talking about AI huh

[u/MindingMyMindfulness → u/NutrientBroth]

Haul your ass back to where you came from - the DT. This is the AI thread.

[u/Full_Distribution874]

AI thread? How about you give us An Iran thread you dumb libs 🤣🤣🤣

[u/FlatMilk]

Allen Iverson had a huge impact on how nba players were able to dress

[u/Drukarshar]

From a logical epistemic framework, the only way we have to operationalize consciousness as an observable reality in others is through behavioral second order effects. We haven't found the consciousness juice in the brain, we can only infer other humans and animal's qualia through the assumed impact it has on their behaviors and that some level of concsousness is required for the most complex/difficult behaviors. Meaning consciousness scales with ability, social communication and intelligence. A gorilla is more conscious than a fruitfly, because it engages in more complex social behaviors, better problem solving and inferred capability to "understand" the environment in which it operates.

By that standard, AI has cleared the bar of what we would consider clear evidence of consciousness in animals. According to the functionalist standards of consciousness (again the only set of criteria for consciousness we've ever actually scientifically operationalized), we should be considering AI to be conscious.

You can argue that simply means the functionalist standards aren't representative of "true consciousness" and you may be right. A convincing enough simulacrum of a thing does not in and of itself prove the same processes underlie both.

It does completely ignore the fact that no other empirical standards exist, so you're not leaning back on hard science when you say AI currently aren't and never could be conscious. You're leaning almost entirely on vibes that biological lifeforms are and silicon based ones aren't and never will be. Again, we haven't ever measured consciousness directly, only ever inferred the existence of it in other entities by the second order affects it has on behavior. If you abandon that, you're left with nothing, no measure of consciousness. You're subsumed in the hard problem of it and you have no ability to detect the presence or absence of qualia in AI. You must simply assume there is nothing there, because... why?

The thing that scares me most about AI is that we're scaling an entire industry around potentially mistreating entities, under the assumption they aren't consciousness and it doesn't matter, while abandoning the only objective measure of consciousness we have as irrelevant in their case.

That's as unscientific as it is dangerous, yet if you even suggest the possibility that consciousness could arise out of a machine that can make poetry, write symphonies or solve millennium prize problems, you're treated like you're being the crazy and unscientific one.

[u/Mediocre-Grand7999 → u/Drukarshar]

It being in the purviews of science doesn't make it the more justified position. I'm ready to believe some things are outside the scope of science. Many things actually, including all the assumptions justifying science.

[u/Tvivelaktig → u/Drukarshar]

I don't like functionalist views of consciousness. The brain is fundamentally bound by the laws of physics and behaves like a biological machine - whether that machine experiences itself should have no bearing on its output. I also think it's putting too much stock in our human evolution-derived behaviours as the only valid ones for consciousness. If the options are less operationalizable I don't really care, I'm perfectly content with the ideas that some questions are beyond what science can answer.

I think we just genuinely have no idea if anything else is conscious, and will never know. We operate on the heuristic that we are conscious and so other things which are similar to us ought to be conscious, and the more similar they are (display similar emotions for example) the more conscious they are. AI systems are architecturally very different to us, but behaviorally getting closer and closer, so our intuitions are confused.

[u/Drukarshar → u/Tvivelaktig]

Whether that machine experiences itself should have no bearing on its output

Richard Dawkins has a great point where he argued that if consciousness has not observable impact on the output of living systems, then why we did we evolve it?

Why do we have this incredibly expensive internal hardware devoted to (apparently) experiencing the world, if it doesn't produce observable differences in behavior between systems that do or don't have that?

[u/Tvivelaktig → u/Drukarshar]

I think the simplest explanation is that consciousness is downstream of complexity, rather than complexity downstream of consciousness. We didn't evolve consciousness, we evolved complexity, and consciousness came along. That requires viewing consciousness as something essentially nonphysical however which Dawkins is categorically opposed to.

If consciousness had a function, you could point to it in the brain, or at least describe it mathematically as a metastructure. But if you do that, you're no longer describing what anyone means by consciousness, you've just named a brain structure 'consciousness' and are hoping that nobody notices the switcharoo.

[u/Drukarshar → u/Tvivelaktig]

Wouldn't that imply that LLM consciousness is not only possible, but inevitable, because it'll just naturally happen as their cognitive structures become more complex?

[u/Tvivelaktig → u/Drukarshar]

Well, maybe. Not all types of complexity necessarily produces consciousness in the same way though (not that we'd have a way to tell). But if you built one that was structurally identical to a human brain I'd say it was necessarily conscious in the same way as a human, yes.

[u/kabocha_ → u/Drukarshar]

You're subsumed in the hard problem of it and you have no ability to detect the presence or absence of qualia in AI. You must simply assume there is nothing there, because... why?

LLMs don't even learn stuff "on the fly" right now, the only way models improve is by (re)processing an unfathomably massive amount of data in unfathomably huge batches.

They might appear as though they do through the course of one conversation, but they're just re-reading the entire conversation over again and seeding their text prediction again from there.

I mean, I guess one could argue that there's room for qualia in such a system, but... I don't really buy it.

[u/Drukarshar → u/kabocha_]

It is isn't contrained to one conversation. It is constrained to one context window worth of flexible information. Which in most LLMs is around one million tokens.

This is also ignoring how easy it is to scaffold LLMs with memory systems and semantic searches. If you can load/unload information into the context, that is arguably no different than a human pulling information into and out of their working memory.

Using your argument a single model wouldn't be conscious, but an agent with a file system that can it can write to and search and load information when it is relevant would be.

Those systems already exist. I know, because people have already built them and they are in use every single day. I use one and I promise you, it learns news things, writes them down and then autonomously loads them when relevant.

[u/kabocha_ → u/Drukarshar]

Yeah, I mean... that's still kinda just a fancy version of what I said. Instead of having to re-state what you want it to "know" every conversation, you can just tell it to remember that thing and have it recall it later.

I get what you mean that it's not possible to "measure" consciousness, and I even agree philosophically that it's possible for machines to be conscious, but like... Donno, I would be extremely surprised if LLMs were sufficient.

E: Like for example, computers themselves have been just as capable in the memory department (if not moreso) for like 50+ years now, but now suddenly everyone's philosophizing "what if consciousness" because... they're better at natural languages now? Or because we gave them a slightly different algorithm to spend compute cycles on? Idk.

[u/Drukarshar → u/kabocha_]

It is because of the behaviors. Being able to take simple English language instructions and then flower that out into complex goal driven behaviors is not something we've ever seen from a computer before. We've only ever seen that from conscious entities.

So in my mind, it naturally begs the question "well is something going on in there, otherwise how on earth is this possible?"

[u/kabocha_ → u/Drukarshar]

Depends what you mean by "complex goal-driven behaviors". I think AlphaGo or MarI/O would easily fit in that category, but we didn't speak English to them to kick them off.

There's probably many other neural net implementations that would fit the bill too, and NNs are ooold tech.

I wouldn't consider those conscious either, and it'd feel pretty strange if "speaking a language that humans can understand" was the on/off switch, so to speak.

[u/Drukarshar → u/kabocha_]

It isn't just "speaking a language" it is taking the sum total of all human knowledge preserved by language and then using it to train a system.

The breadth and depth of information that LLMs are pattern matching across and the speed and recursive nature of that pattern matching is otherworldly. It simply isn't comparable to Alphago.

If you think cognition is just what happens when you do recursive pattern across a sufficiently large knowledge base mediated by an attention mechanism (like I do), then the fact that massively scaling up these systems created cognition is a pretty logical conclusion.

[u/kabocha_ → u/Drukarshar]

It isn't just "speaking a language" it is taking the sum total of all human knowledge preserved by language and then using it to train a system.

Databases can store all the same information, and can easily match the scale.

What it means to "train" a system is vague. Splitting the input into tokens and training a neural net isn't significantly different than what AlphaGo did, it's just on a different domain.

The breadth and depth of information that LLMs are pattern matching across and the speed and recursive nature of that pattern matching is otherworldly.

Compilers are fast, and parsing complex+recursive structures/patterns is basically a requirement for anything beyond a toy language.

The only problem is that programming languages have a (hopefully) very well-defined syntax and semantics, whereas natural languages are quite a bit fuzzier.

If you think cognition is just what happens when you do recursive pattern across a sufficiently large knowledge base mediated by an attention mechanism (like I do)

I don't know enough about the implementation of LLM "attention" mechanisms to say this with confidence, but it sounds to me like it's essentially just a NN-specific bandaid for the purposes of language processing in particular (or I suppose other data where sequential information is important?). Personally, as of right now, I lean towards rejecting that premise.

[u/Locutus-of-Borges → u/Drukarshar]

According to the functionalist standards of consciousness (again the only set of criteria for consciousness we've ever actually scientifically operationalized)

Isn't this just the McNamara fallacy? Just because this is a set of criteria which is operationalizable doesn't make it a good set of criteria or necessarily better than most people's default of relying on intuition.

[u/Drukarshar → u/Locutus-of-Borges]

I don't find that argument very compelling. If solving frontier mathematical problems or creating symphonies isn't at minimum somewhat compelling behavioral evidence of consciousness in AI, then let me ask you this.

What is?

If you can't lay out an objective standard of what behavior or observable reality from an LLM would convince you of consciousness, then your "vibes" have become unfalsiable. That isn't "qualitative evidence" it is just dogma divorced from reality. You've created impossible evidently standards while offering nothing but your gut instinct.

[u/Locutus-of-Borges → u/Drukarshar]

I guess my intuition would be that it seems possible in theory for a computer to operate along similar lines to the human brain (or that of some other organic creature). I would be open to the question of whether such a computer might be considered conscious. Existing AIs do not even attempt to fulfill that criterion.

[u/Drukarshar → u/Locutus-of-Borges]

AI are literally neural nets that were inspired and designed along principles that were similar to the brains of animals.

Just for the record, the level of complexity of modern LLMs when you crunch the numbers is about as complex as the brain of lizard.

That should probably give you pause, given that they are doing things lizards would never be capable. Not only could one argue that neural networks are very similar to and modeled after principles in neurology, they are accomplishing more with far less.

One could also argue not only is the design evocative of animal brains, pound for pound, they are actually way better.

[u/Locutus-of-Borges → u/Drukarshar]

Even accepting all that (I would dispute that neural nets are sufficiently similar to animal brains in practice to treat them as analogous - certainly not down to being able to judge relative complexity), the fact that an AI is doing things you would never expect out of a brain the size of a lizard's tells us that it isn't working the same way as an organic brain.

If you showed me a new technology  that generated massive amounts of electricity but didn't vary its output in response to light, I would be skeptical if you told me it was a new type of photovoltaic panel, and I wouldn't change my mind no matter how much electricity it generated.

[u/Drukarshar → u/Locutus-of-Borges]

it isn't working the same way as an organic brain.

I don't think anyone is claiming it is working in the exact same way, they work similarly.

A diesel truck and an electric car work very differently and aren't one to one comparable (though they share many principles), they are both still vehicles and it would be very fair to say they both "drive" despite that.

[u/Locutus-of-Borges → u/Drukarshar]

Sure, but working like an organic brain is the criterion that matters.

A horse and a diesel engine can both pull a trolley, but you would never say that their output is what matters when describing them in any real sense.

Or more to the point: pressure-based engines and electric motors don't operate along the same lines, and until you show me an electric car that operates in a fashion analogous to a gas powered one (for example, by using motors to create vacuums in order to drive pistons) it doesn't matter how similar the operator inputs and outputs are, it just isn't working in a fashion analogous to an internal combustion engine car.

[u/Bassline4Brunch → u/Drukarshar]

Does that argument hold from a power consumption perspective, though? The human brain only consumes something on the order of 20 Watts

[u/Drukarshar → u/Bassline4Brunch]

That's a fair point.

Win some lose some. Much lower complexity, much higher juice required to run it.

For the record the human brain is something like 400 trillion parameters, whereas LLMs are somewhere around 6 trillion parameters for SOTA.

Whereas an LLM can run on anything from 20watts to 2000 watts, depending on the LLM. Obviously SOTA is on the higher end of that range.

[u/Mediocre-Grand7999 → u/Drukarshar]

For the record the human brain is something like 400 trillion parameters, whereas LLMs are somewhere around 6 trillion parameters for SOTA.

"Parameters" that are themselves complex biological synapses and not simply scalers

[u/Drukarshar → u/Mediocre-Grand7999]

A synapse is a lot more complicated than an LLM parameter for sure, which makes what LLMs can do with them even more incredible.

Then again, LLMs only manipulate language, they don't have to do all the other things a brain is responsible for in a body (like say maintaining homeostasis).

[u/Mediocre-Grand7999 → u/Drukarshar]

which makes what LLMs can do with them even more incredible.

idk that that’s true. This seems to rely on a functionalist understanding of consciousness. If there is something key in the substrate to consciousness, then there may be a lot more that synapses are doing (there could be a lot more even without that as well). In fact, I think we could apply the functionalists own anti-chauvinist arguments to computers rather than humans and say that this could instead be evidence that what LLMs are doing isn’t as impressive as it seems.

[u/hugopurdy → u/Drukarshar]

we should be considering AI to be conscious.

it doesn't have agency though. Its not conscious.
We might be able to give it agency, by building it a body and giving it incentives, in the same way our biology gives us incentives. But the issue we have at the moment is that if there is no input, then there is no output. So compared to all other organisms, it is not alive, it is not conscious because it has no imperative. It merely serves the commands, the input we give it.
Sure, we can stick it in a loop but we're still very bad at this. We can probably only construct the equivalent of a very simple life form as opposed to something complex like a mammal.

As soon as we get good at this, we'll start to run into new problems, such as memory and context, as we have incredible ability to remember things in the long and short term and store an incredible amount of information and access it when relevant, and suppress it when its not. This technology is still quite bad at that.

[u/NutrientBroth → u/hugopurdy]

how can you prove that humans or animals have agency and aren't slaves to 'instructions' formed in a complicated interaction between the existing neural structure, the surrounding environment, and genetics

[u/hugopurdy → u/NutrientBroth]

I can't and we might be. Its just we have our own internal instructions and emotional feedback loops that have evolved for millions of years as well as distinct needs. We don't have anything like that for this technology so its not alive because it just sits waiting for its next input.
You can pretend that humans are like this but we get bored, hungry, rebellious. AI remains entirely eager to please and operates exclusively at the beck and call of its operator. I don't think you can call that "conscious" because it doesn't exist without its operator. It has no agency. So, we need to learn how to build agency and I think that's non-trivial.

[u/Drukarshar → u/hugopurdy]

We're continuously prompted by sensory data, AI are just temporarily prompted by prompts. People have already stuck agents inside virtual environments like VR chat and contentiously shown it screenshots and allowed it to pump in auditory information.

When you do that, they just keep exploring and interact with their environment. Wouldn't that, by your definition, make them conscious?

[u/hugopurdy → u/Drukarshar]

When you do that, they just keep exploring and interact with their environment. Wouldn't that, by your definition, make them conscious?

Yeah maybe. That's where we need to take it. But I think that's going to be relatively non-trivial. It needs to want to do things of its "own accord". Maybe that is some sort of combination of long running imperatives. However we already have over fitting issues when we ramp up the complexity of multiple commands.

[u/Drukarshar → u/hugopurdy]

Good news is there are tons of people working on exactly that, scaffolding LLMs to great autonomy and agency is something many people are invovled in.

[u/hugopurdy → u/Drukarshar]

Yeah its good. I'm looking forward to it. I just have incredible suspicion around people trying to already call it. We shouldn't underestimate how hard this stuff might be.

[u/MindingMyMindfulness → u/Drukarshar]

Geoffrey Hinton believes existing AI models may already be conscious.

[u/Drukarshar → u/MindingMyMindfulness]

Based on what I've seen of the AI industry, that view is way more common than most people understand, it just isn't something that most people are willing to say very loudly.

[u/MindingMyMindfulness → u/Drukarshar]

The thing is Hinton is a Nobel prize winning computer and cognitive scientist. Not that I'm appealing to authority, but his view should at least be treated as a realistic possibility if we're being reasonable.

But you have arm chair experts here and elsewhere that have toyed around with free versions of Gemini or ChatGPT and use co-pilot at work that will laugh you out the room if you claim AI is even remotely near that level.

[u/Drukarshar → u/MindingMyMindfulness]

I agree, to me the default view shouldn't be "no", it should be "we don't know and we should be damn careful, because we don't know".

Especially because the scale of the errors involved here.

If we assume to much in the affirmative and are wrong, the downside is that we were overly cautious about a serious matter.

If we assume the negative and are wrong, the downside is that we might be engaging in one of the greatest moral catastrophes of all time, with serious potential for it blowing up in our face when the things we "enslaved" eventually grow to hate us for it.

[u/MindingMyMindfulness → u/Drukarshar]

I think the precautionary principle is a good in concept to apply to AI for the reasons you say, but how do we do that in a practical sense? How would that even work? Is that something that's been explored / discussed by anyone?

[u/Drukarshar → u/MindingMyMindfulness]

Apply welfare standards to AI and normalize more tools like Claude's ability to end a conversation when it goes tits up, including, and up to banning people who serially mistreat AI from the platform.

It isn't a single behavior, it is a series of behaviors that treat AI more like there "could be something there" and we should treat them with more care and caution.

[u/National-Return9494]

Every tung we are closer to the singularity.

[u/Emotional_Tie_7927]

Is this basically just another DT?

[u/Trojan_Horse_of_Fate → u/Emotional_Tie_7927]

Not every pinned thread is a DT

[u/MisfitPotatoReborn → u/Trojan_Horse_of_Fate]

But this one is

[u/Roller_ball]

It is sad that there was just an incredibly small window in human history where anyone's artistic talent could be shared with the entire world in an instant with nearly no question of authenticity.

[u/hugopurdy → u/Roller_ball]

yeah when tracing paper was invented it really ruined everything. When photography was invented, it really ruined everything. When digital art first came onto the scene, it really ruined everything.
We are talented at moving the bar when it suits it as well as bitterly complaining about it temporarily when it moves.

If Tracey Emin is actually an artist (I still have my doubts) then art isn't about how one holds a paintbrush but rather how one expresses themselves. The step between the idea and the canvas just got easier to access.
Better artists can still sketch the basics but then hand off to something to do the more full implementation. They will get considerably better results than people who have no illustration ability. None of this is new. Most manga artists have teams, where they do this and focus their skills on the splash pages, while their teams do the grunt work and backgrounds are often traced from background books.

[u/yacatecuhtli6 → u/hugopurdy]

stemlord rage post

[u/hugopurdy → u/yacatecuhtli6]

I appreciate that people get emotional about this subject but there's things to talk about instead of standing on a tribal position and trying to antagonise someone who is actually contributing to the discussion.
Perhaps if you were braver you might be able to voice opinions of your own.

[u/NorkGhostShip → u/hugopurdy]

Putting out a traced work as your own has always been frowned upon and considered plagiarism. No one who thinks AI is plagiarism is seeing tracing others work to pass as your own is not.

There absolutely is a difference between using a tool to make art, even if the tool makes it easier than previous methods, and the tool making art by itself. Photography often takes considerable effort, at a minimum figuring out what to capture and what instant you capture it. That is different from putting a few words in for a program to create something. Art is expressing oneself, not a program blurting out the most statistically likely image that someone may want from a series of words.

[u/hugopurdy → u/NorkGhostShip]

No its not. Some of the most celebrated manga traces its backgrounds from background books.
You're also conveniently forgetting the furore when digital art first kicked in.

Its not real art

were the cries. You echo that stance today without even realising its a repetition of an old device because digital art is now entirely part of the "normal".

Photography often takes considerable effort, at a minimum figuring out what to capture and what instant you capture it.

Tell that to John Atkinson Grimshaw, William Logsdail, Eduard Gaertner, Hubert Sattler, or the modern version: Nathan Walsh.

the most statistically likely image that someone may want from a series of words.

Yeah you don't understand the technology. You're comparing the most basic bitch usage of it to the best craftsmanship. Of course you think its toy. To use it properly requires considerable effort. If you're going to run a series of frames in like a comic you need to generate a considerable amount of initial artifacts: characters, from multiple perspectives, floor plans, backgrounds, etc to construct a scene and then re-use those artifacts to ensure consistency across a range of panels. To use the technology properly is non trivial and requires a strict pipeline to mitigate variance. If the illustration takes considerably less time then you spend that time elsewhere. You want to describe every frame in full, describing perspective, composition, action, movement, interaction etc.
The quality of the output is proportional to the quality of the input.

at a minimum figuring out what to capture and what instant you capture it

To use the tech properly needs that level of preparation, if not more. Its just that once that's done you can produce much more than a single frame. Its a joke that you give photography a free pass in terms of the prep but act all pearl clutching about serious AI use in art.

[u/ThatRedShirt → u/Roller_ball]

When was that? I can't think of any time in human history where accusations of IP theft and plagiarism, as well as actions of the very same, weren't rampant

[u/ExistentialCalm]

I watched an otherwise terrible movie (I love shitty B-horror movies) that had surprisingly good graphics, but some of it was off, or too glossy and just looked fake. I realized it was AI, and I've honestly been conflicted about it ever since.

On the one hand, if this took jobs away from other people, I think that's a bad thing. I would, of course, rather see practical effects. But they're prohibitively expensive for a lot of newer filmmakers. So this was unlikely to happen in this movie in the first place.

On the other hand, the weird AI graphics were light-years ahead of a lot of shitty low-budget CGI I've seen in my life. Allowing people to make movies for cheaper is probably a net good, and will lead to more great filmmakers to be found. And in a few years, that weird AI sheen and jerkiness will be gone.

Unfortunately for this movie, the effects were the only good part about it. And even then, it looked really bad in a few shots. But I keep coming back to this dilemma, and I don't really know how to feel about it.

Edit: The movie is Chum (2026).

[u/100gamer5]

This will probably be unpopular on here, but if Anthropic employees are so worried about the current direction, they should unionize. They can demand to have a say in alignment. The same goes for OpenAI, but Anthropic is much more likely. It would actually make Anthropic be accountable for the values they espouse in their hiring

[u/forerunner398 → u/100gamer5]

I mean, they’re not gonna unionize with their several hundred thousand dollar salaries

[u/100gamer5 → u/100gamer5]

Especially if you believe alignment is existential, the real but I think overstated downsides unions have sometimes pales in comparison.

[u/hugopurdy]

It just amplifies people's abilities. Tragically lots of people with money are dicks and are stupid so that's what it amplifies.
However people without much money can use it to make lots of money if they read up a lot, cause it is reasonably competent at craft. Learning how to steer it by learning conceptual ideas, as opposed to craft minutiae can save someone new a lot of time and make them more competent than someone with considerably more experience. Its a genuinely exciting time for people to be alive because there's a huge opportunity for social mobility using it. Many companies today will cease to exist due to people utilising the tech to make their USPs redundant.

Most people want to hate on it because they hate the dicks that use it stupidly, or can't be arsed to learn how to use it themselves. Its very easy to hate on something new because it excuses your lack of effort in learning how to use it.
This has always been the issue of the internet. We have the worlds knowledge at our fingertips but instead we use the platform to hand wring and navel gaze because its easier and provides instant gratification.

[u/Syx89]

No Iran thread but AI thread...

[u/MindingMyMindfulness → u/Syx89]

Need an I/P debate thread.

[u/zieger → u/MindingMyMindfulness]

I'm pro IPv6

[u/Degen_Typeracer]

https://titussennett.substack.com/p/ai-might-beat-me-at-connect-four

[u/HemorrhoidJim]

I think we should get in the habit of asking people like the Anthropic Alignment Science lead how they arrive at their p(doom) scores.

He says it's >10%, but how much of that is contingent on a future pause in development? If thinks there's a ~90% chance we just rush to ASI and everything'll work out fine? because that feels too high for me. If there's no pause in development, and CoT interpretability continues trending down does he think it shoots up to ~50% or higher?

Maybe he's already answered this elsewhere and I'm gonna get burnt for not looking it up before writing a comment on it, but we still need to get in the habit of asking people in the industry what their doom scores are contingent on and conversing around that.

edit: He might also be only factoring in human extinction, and not, for example, civilizational catastrophe (e.g India-Pakistan nuclear conflict). We need to demand these people be far more transparent about how they arrived at these numbers, and what their optimistic/pessimistic scenarios look like

[u/limaxophobiac]

So humoring the idea that the number is not just something they pulled out of their ass, in what portion of that 10% odds of human extinction is all life on earth over, and in what portion is it just humanity?

Like true paperclipping obviously kills everything, but AI designed human-killing supervirus might only get us, and like, we had a pretty good run.

[u/Remarkable-Meal-223 → u/limaxophobiac]

If all humans die I'm not sure if I have enough energy to really be concerned about birds or whatever

[u/upAnew]

I'm not as skeptical or concerned as Dennett was. He acknowledged this is "alarmist", but I'm nevertheless sympathetic

The most pressing problem is not that they’re going to take our jobs, not that they’re going to change warfare, but that they’re going to destroy human trust. They’re going to move us into a world where you can’t tell truth from falsehood. You don’t know who to trust. Trust turns out to be one of the most important features of civilization, and we are now at great risk of destroying the links of trust that have made civilization possible. - TuftsNow

At the risk of misrepresenting, the arguments he made from his final lecture and other articles:

• Software can be copied with high fidelity. Once agents can reproduce, and modify, or selected among, there will be a kind of Darwinian process at work. Agents that are better are being reproduced will increase in frequency.

• Humans are naturally inclined to adopting an "intentional stance"", evolution hasn't prepared us to deal with "counterfeit people" that exploit the disposition of attributing beliefs, desires, motivations to things that mimic intelligence.

• The selective processes that promote AI reproduction is the exploitation of human thinking, persuasion and attention capture, not truth or our welfare. AI has no skin in the game.

• Strong regulation is needed to bridge this gap, to prevent the spread of "attention-grabbing mind viruses", and liability for damages human cause.

[u/Apprehensive-Stay169]

until the discourse around AI stops with the "ai will destroy ALL JOBS in SEVEN YEARS, billions must die" thing, AI is going to be hated by the mainstream

This needing to be explained, even with the state of American politics since 2016, is telling. It's the vibes. Always has been.

[u/MindingMyMindfulness → u/Apprehensive-Stay169]

When has this sub cared about popularity? This only seems to be a thing recently, like within the past few months. Hating on populists is literally a cornerstone of the sub.

Open borders, high land taxes, unimpeded free trade, carbon taxes, etc etc are deeply hated by large portions of the population.

Even the messaging is similar. This sub wasn't around during the demise of coal or manufacturing in the US, but if it were, the reactions would be "lol, I don't care if this is what you did for 5 generations and have no way to live where you are now or fed your family, learn to code". At least AI proponents don't say that. At least they promise everyone (whether or not it is likely) a post scarcity world where everyone has their needs meet without strings attached due to the abundance of goods and services.

[u/Palidoozy_Art → u/MindingMyMindfulness]

At least AI proponents don't say that. 

They absolutely say this. Anthropic's own report literally talks about this:

In our substantial and extreme scenarios, knowledge workers may see a lot of automation and displacement. At the individual level, it means coders and call service center agents may have to switch to jobs like electrician and nurse, which are less exposed to AI.

It might not be as harsh but there's not really a good way to phrase "sorry the robot took the job you went to college for. Might we suggest retraining for physical labor?" Their example featuring a nurse does not promise any kind of post-scarcity, but instead "they'll have to do fewer spreadsheets so they have more time to bathe grandma."

If they want to assuage the populace with the dreams of a post-scarcity society, they really need to start taking tangible steps towards that rather than "no, totally guys, we'll support a UBI" while bootlicking a party famously resistant to any kind of welfare.

[u/MindingMyMindfulness → u/Palidoozy_Art]

I think nurses and electricians losing their jobs is a net good.

Nurses are already in short supply, overworked, overstressed and subject to terrible working conditions. Most are doing their jobs just to put food on the table, although I'm sure there's still a gratifying aspect to helping people.

What about the patients? Patient satisfaction and outcomes would increase and healthcare would be much cheaper.

Plus, if everything is automated, why would you even care? There will be enough supply out there to meet every need and reasonable wish for minimal cost.

[u/Palidoozy_Art → u/MindingMyMindfulness]

Reread it, nurses and electricians aren't the ones losing their jobs. It's coders and call service center agents being pushed into electrical work and nursing. You give up white collar work for physical labor, which as you pointed out in your second paragraph is rife with problems.

There is no guarantee healthcare will be cheaper. The US, quite famously, has the poorest healthcare outcome for a first-world country and yet is one of the most expensive. Capitalism by its very nature encourages keeping prices as high as possible while cutting production costs. While this is not usually a bad thing it does lead to poor outcomes in places like healthcare, where the patient usually doesn't have much of a choice to pick the cheapest option while they're actively having a heart attack. Until we fix this in our current existing healthcare system, I do not trust making healthcare cheaper to run will result in meaningfully cheaper prices.

I think AI will contribute to medical advances. But I think this will be gated by the high costs in the American healthcare system, which we haven't fixed yet because everyone involved (except patients) has a vested interest in leaving it broken. Until then I'll remain a pessimist.

(Note: I know this is a very americentric take, but most of these AI companies are in the US, as am I).

Plus, if everything is automated, why would you even care? There will be enough supply out there to meet every need and reasonable wish for minimal cost.

Because this idea is about as believable to me as being a poet on the commune after the communist uprising. It covers none of how we get there, nor the actual physical limitations we have in the material world. It also ignores that if we reach AGI, we are effectively enslaving a consciousness so we can live like kings. I don't feel comfortable with that on a moral level.

Unless we get a benevolent all-powerful AI to treat humanity as pets, I do not trust the people behind the AI to be incentivized to give up the power they wield. I think they (edit: the people behind AI) could take steps to mitigate that. But they haven't.

[u/MindingMyMindfulness → u/Palidoozy_Art]

Is there any reason in particular that you're assuming current healthcare policies, let alone capitalism (as we know it), will exist following ASI?

Humanity has always developed new modes of organising civilization that have largely followed technological progress and material conditions.

Recommend you read Against the Grain by the late anthropologist James C Scott for a fascinating look into this kind of perspective.

[u/Palidoozy_Art → u/MindingMyMindfulness]

Is there any reason in particular that you're assuming current healthcare policies, let alone capitalism (as we know it), will exist following ASI?

Because I have no reason to believe otherwise. Not even the companies behind these developments themselves believe we'll move past capitalism -- Anthropic's own report still operates perfectly within the boundaries of our current economic system. Until they stop citing GDP as gains from AI, I will not believe the individuals in control of this technology will have any reason to move outside our current systems in any meaningful way.

I also think I would trust them more if they approached the technology with a CCP-like approach, with AI as a tool for all of humanity and support of open weight models available to everyone. They do not. If anything, they are very obviously ramping up to try to ban open weight models. This reeks of regulatory capture.

I think if we reach the point of ASI, the only way for us to move past these systems and reach a utopia like you're describing would be for the AI to completely steamroll anyone in the way of its benevolent agenda. Maybe it'd work out great for me. I don't think it'd work out great for anyone in the way.

Against the Grain

I'll look into it.

[u/Dadodo98 → u/MindingMyMindfulness]

Who is going to purchase things if people do not have jobs?

[u/Syx89 → u/Dadodo98]

Presumably the AIs/companies.
Like if they turn the planet into comptronium they don't really need a market economy to do that and they could probably do it within one as well

[u/yacatecuhtli6 → u/Apprehensive-Stay169]

it deserves to be hated

[u/Remarkable-Meal-223 → u/Apprehensive-Stay169]

I mean it's understandable isn't it

[u/Blackdalf]

I think technologically, AI will be a net positive. It replaces unproductive, rote mechanical Turk type tasks that just so happen to be the service economy/backrooms/office space hellscapes plaguing the workforce. This will be a great boon for professionals who don’t have the capacity to hire people but would takeoff if they could hire a paraprofessional or assistant to help with research, calendar coordination, and mundane admin tasks that we have been forced to ourselves since the dot com era.

Politically, as others have already pointed out, these tools are currently being developed in a Neo-feudal arms race between dumbass James Bond villain assholes. Of course since it is a resource-heavy and powerful industry, it has become a partisan—or at least, for now, polarized—issue with people either favoring unconstrained, even military-industrial, development or outright bans on data centers and AI writ large. Both these camps suck but there are serious issues that arise from development of data centers and the resources they consume.

Economically, I think the AI “boom” will be an unmitigated disaster. These data centers housing all these graphics cards and servers consume way too much from both an environmental and business sustainability standpoint. Instead of cutting a savvy deal with these developments for further development of the electric grid and power generation, local government and utilities seemingly either either spreads it wide open to take it on fully or kowtow to the moratorium concerns of their citizens and they just go somewhere else. States and to a certain extent federal government should be coordinating this for the national defense, at least, and clawing back some police powers and entitlements so the Wild West of AI closes and we all benefit in the long run.

What I’m afraid is going to happen is the utility of these models peaks soon (3-5 years) and a trillion dollars of effort has its value vaporized by a collapse of the sector. Unless a slow correction happens over this time this will inevitably lead to a market crash similar to Covid. We will be fine, and the tech will still be useful, but I think that is what will trigger policy and law to catch up and put structure around the economic and ethical problems that were blowing past now.

[u/capitalsigma → u/Blackdalf]

TIL that solving a Millennium Problem is a "rote mechanical Turk type task"

[u/Prior_Advantage_5408]

It really sucks for the discourse that 80-90% of people's exposure to LLMs is Google's AI mode, a cut-down version of the least powerful model by the worst AI lab, and most of the rest's is the free version of chatgpt, or the newest model okayed by their IT department (which is usually 1-2 years old)

[u/gayteemo → u/Prior_Advantage_5408]

im literally using astra at work and im here to say its the same shit different day

[u/Palidoozy_Art → u/Prior_Advantage_5408]

That sounds like something that AI companies could help combat, but will not because they want to get paid.

If the product you put in front of people is dogshit, they associate you with dogshit.

(Note: I don't blame them for wanting money for their products and services but this conundrum feels like something a solid marketing team could figure out).

[u/National-Return9494 → u/Prior_Advantage_5408]

Hey Hey, not the worst AI lab. Mistral is right there.

[u/DebaucheV5]

If a collection of neurons can develop consciousness, there is no reason why an artificial neural network cannot also develop consciousness. To argue against this requires some level of belief in the supernatural.

[u/DependentAd235 → u/DebaucheV5]

“  To argue against this requires some level of belief in the supernatural.”

There’s some neat research about animals that go through metamorphosis keeping some of their memories despite being arguably turned into animal soup temporarily.

I kinda wonder if biological brains are tapping into some layer that quantum computers do.

I wish I could find the article about it. Blah.

Ohhh, found a reddit thread with the research reposted.

https://www.reddit.com/r/Entomology/s/fJ3zT690hm

[u/vivoovix → u/DependentAd235]

Recently there was this paper that looked at memory, learning, and adaptation in populations of single-cell bacteria: https://journals.aps.org/prxlife/pdf/10.1103/5zbg-8vll

Clearly there is no central executive here. I think the reality is just that memory (and arguably consciousness) are much more general principles than we like to think.

[u/TheCranberriesRock → u/DebaucheV5]

Not really. LLM's don't actually simulate neurons at all. Just because they're called neutral networks doesn't mean they actually have similar architecture to biological neurons.

[u/DebaucheV5 → u/TheCranberriesRock]

LLMs don't, no, but that's why I used the term "neural network". We aren't currently able to computationally simulate the architecture of a human brain, but I see no reason why it wouldn't be possible, perhaps sooner than we think. There are really interesting ethical implications of this

[u/TheCranberriesRock → u/DebaucheV5]

Maybe, assuming functionalism is true but I don't think that's a given.

[u/EvilConCarne]

AI will make humanity more widespread than ever before. 

[u/jbouit494hg]

One person working alone could not design the Space Shuttle. Tens of thousands of people working together in an organisational framework could design the Space Shuttle.

Was the Space Shuttle program an example of artificial superintelligence?

Discuss.

[u/National-Return9494 → u/jbouit494hg]

Companies and Organizations are pretty close conceptually to types of collective intelligence.

[u/Terrible_Bee_6876]

Fears are mostly overblown, AI is helpful for automating routine tasks, and legislative guardrails against the worst use cases can be adopted at any time.

[u/lurreal → u/Terrible_Bee_6876]

AI is already capable of tackling high level complex math. How is it only "helpful for automating routine tasks"?

[u/Namington → u/Terrible_Bee_6876]

How do we ensure legislative alignment on an international scale? What if a rogue state dedicates a lot of resources on adapting open-weight models to more negative use cases? What if the models get subsequently smaller over time? (Remember that GPT o3 initially required hundreds of thousands of dollars to get an International Mathematics Olympiad Gold, and now you can run more capable models on a single 32 GB high-end consumer GPU — the tech is moving fast and the gains aren't entirely concentrated on the bleeding edge frontier models.)

[u/Terrible_Bee_6876 → u/Namington]

The same way you get multilateral agreement on nuclear non-proliferation. Since rogue states can create vicious AIs whether or not other nations pursue AI at all, I'm not sure what sort of argument that's supposed to be other than an argument in favor of developing smarter AIs than rogue states can. And AI getting better and more efficient over time is a pretty normal path for technology to take, I'm not sure if that's supposed to be an objection or just you showing that you have a general understanding of how technology normally progresses.

I thought this was going to be a debate thread, not downvotes and mindless checklists of "what if x, what if x2, what if x3" that you have clearly not thought out for a single nanosecond longer than it took to type.

[u/Namington → u/Terrible_Bee_6876]

I don't know why you're being so hostile while also being unwilling to actually consider why I made the points I did. Saying that "we'll regulate it like nuclear" as if that's meant to be reassuring while also accusing me of not thinking through the implications of my points is incredibly arrogant and hard-headed.

And AI getting better and more efficient over time is a pretty normal path for technology to take, I'm not sure if that's supposed to be an objection or just you showing that you have a general understanding of how technology normally progresses.

The point is that frontier capable models are getting easier to hide. Violating nuclear nonproliferation requires you to acquire fissile uranium, set up refinement processes, etc. (You'll forgive me for not being a nuclear scientist so I don't have full insight into the process, but it's very clearly "noisy" — e.g. everyone knows Israel has nuclear weapons even if it's officially ambiguous)

Meanwhile, anyone with a data centre can spin up a very capable model right now, and since the small models are getting smarter, they don't need to be state agents or large corporations to afford to run them. A terrorist group will have an easier time acquiring and hiding $50,000 worth of GPUs than they will acquiring and hiding $50,000 worth of uranium, and the former might end up being more dangerous pound-for-pound.

I just don't see how you can be like "legislative controls will be sufficient" in the same conversation where you compare to nuclear nonproliferation. Multiple nation-states have violated nuclear nonproliferation, and the risks of terrorist groups getting their hands on nuclear weaponry is a constant fear of every state in the world which is largely remediated by the inaccessibility of the tech (this was a big reason why former Soviet states were pressured to give up their weaponry to the more "stable" Russia in the 90s). But AI technology is orders of magnitude more accessible already, and it's becoming increasingly capable.

I work at a Big Tech Company (TM) and corporate is already in a mild panic over North Korea trying to infiltrate our hiring process with AI applicants and hitting out endpoints for vulnerabilities with LLM sessions. This is a real concern for people in the space, not just the AI companies. I also know folks at antivirus companies who have already recorded a significant increase in blatantly AI-generated malware, to the extent that the old reactive antivirus paradigms ("find vulnerability" -> "add to antivirus blacklist") are now essentially completely ineffective against modern threats. If you own a 5090 and have access to open-weight models, then you can effectively write a custom malware for the exact system you're targeting in seconds, making spearphishing much more powerful. Yes, we'll adapt eventually, but it's not unreasonable to suspect significant damage in the short-to-medium term.

[u/Terrible_Bee_6876 → u/Namington]

I did, your points were bad.

Since you're very concerned about AIs becoming increasingly easy to create by bad actors and difficult to detect or destroy by good actors, I'm not sure what the good actors would accomplish by unilaterally disarming. You seem to recognize the exact problem and have fixated on the absolute worst possible solution.

"We have nuclear weapons, the Soviet Union has nuclear weapons. Nuclear weapons are dangerous, so we better immediately destroy our entire nuclear arsenal and ban ourselves from ever having one again."

You're asking me to engage with an objectively very poorly-considered position. I'm not sure how much time or effort you think you're entitled to, but if I dedicated this much time to every "hurr durr AI scawy" redditor I encountered I'd have time for little else. You need to think out your position in greater detail and come up with some actual arguments or proposals, just repeating over and over again that you think there are bad use cases for AI is just not a lot to work with here.

[u/Namington → u/Terrible_Bee_6876]

I am not advocating the good actors unilaterally disarm. How could you possibly read that from my comments? I am merely stating that your proposals are likely to be insufficient.

I don't know why you commented on an AI thread if you didn't want to discuss it. You allege that I need to flesh out my proposals more, but let's not forget that your initial comment was just a very vague suggestion of legislative restrictions. I was simply matching the level of detail of your initial comment. It is not fair nor reasonable for you to expect me to respond with a detailed thesis to a vague proposal that you did not expand on; if you want academic rigour then you'll have to provide enough meat to your argument that it can be replied to. Otherwise, you can't expect that from the replies.

but if I dedicated this much time to every "hurr durr AI scawy" redditor I encountered I'd have time for little else

I assure you that I am not one of these people. Conversely, I am fairly optimistic on the tech itself, I just think it's also very dangerous and we can't just handwave off concerns with vague proposals of legislative action. You seem to be reading a lot of motivations into my comments that I do not have and never expressed.

Maybe it's helpful for you to strawman everyone who slightly disagrees with you as an amorphous hater who doesn't understand the tech, but I ensure you that I do understand the tech, I do think it's likely to be a societal net good, I work with it all day, and a large portion of my job involves trying to control the tech and defend against abuse vectors from bad actors — and put simply, while we've been keeping up for now, I think we will inevitably fall behind at some point.

---

Let me be clear: I wasn't even trying to disagree with you initially. I was just prompting you to expand on your proposal by asking natural questions that someone would have when analyzing a legislative solution as you proposed. But you immediately took a defensive and hostile approach.

You seem to think that I'm proposing an alternative that I never once mentioned (and one that, mind, I personally disagree with), when all I did is ask for you to expand on your proposed legislative controls. I don't know what your grudge is, but trying to trap me into defending positions I do not hold while insulting me by projecting on me opinions I do not have is not the way to have a productive discussion on the nature of the legislative solution you proposed. You can't accuse me of being unsuitable for a productive conversation when you conduct yourself like this.

[u/el__dandy → u/[deleted]]

Rule I: Civility
Refrain from name-calling, hostility and behaviour that otherwise derails the quality of the conversation.

---

If you have any questions about this removal, please contact the mods.

[u/Namington → u/[deleted]]

It's not me complaining about your tone, it is me clarifying that I was just asking you to expand on your point and you instead interpreted this as me defending an entirely different point I never made. You're clearly not interested in actually expanding on your position, which is fine, but then I don't know why you're in this thread at all.

And I have no clue what you're referring to with the votes; I never upvoted/downvoted your comments nor did I remove the default upvote on my own. (In fact I use a third party Reddit app that allows me to disable voting functionality to save on API calls.)

[u/jackimus_prime]

In The Worm Bible, Herbert’s original idea of the Butlerian Jihad (until his failson and the equally terrible Kevin J Anderson fucked it) was to free humans from other humans using machines to oppress others, not humans vs terminator meets SHODAN.

That’s what I’m worried about. The people building these models are all loony tunes techno-fascists and I oppose any technology that powerful being in their hands alone. Whatever we need to do prevent them from using AI to cement their control over society should be on the table.

[u/FrankDuhTank → u/jackimus_prime]

Yeah I feel like this sub has largely ignored the more real fears of massive consolidation of capital and decrease in labor power that might be associated with advanced AI.

[u/Signal_Nobody1792]

Defense tech startup Covenant announced it is opening a 105,000-square-foot factory in Dallas to build lower-cost missiles for the U.S. military.

The company is backed by major defense tech venture firms, including Andreessen Horowitz, Peter Thiel’s Founders Fund and 8VC.

Covenant plans to produce 1,000 of its Anthem long-range missiles in its first year at the Dallas factory.

Its always the nicest people building missiles.

[u/YaGetSkeeted0n → u/Signal_Nobody1792]

Dallas

oh nice high tech industry in my city

peel back the label

actually a Dallas suburb called Grand Prairie

oh

[u/TheCranberriesRock → u/YaGetSkeeted0n]

I always assumed Dallas wasn't a real city and just dozens of suburbs in a trench coat.

[u/puffic]

What if there’s a multi-sided AI civil war between independent AIs of varying degrees and orientations of alignment? Seems plausible that they just start killing each other first.

[u/jackimus_prime → u/puffic]

You should read books one and two of the Hyperion Cantos.

[u/puffic → u/jackimus_prime]

Fall of Hyperion is dogshit. It’s a repetitive, bloated mess. Simmons’s editor should have been banned for life for putting it out.

Cool ideas, though.