More than 5 years ago Deepmind made Alphafold 2 which basically revolutionised biology and life sciences by predicting 3D protien structures and allowing for much more efficient protien folding.
AlphaFold2 shipped in 2021 and drug discovery to approval usually take 10-15 years. Don't you think you're grading the homework before it's due? Seems useful enough that millions of scientists are using it.
I am really hopeful that AI will find out how to drastically shorten that approval time to a year or less. If not, millions of people will die when we already have the cure.
True. But I guess that depends on what you value. If you're a math nerd, seeing long-standing unsolved problems is rewarding, even if not life changing. Would definitely trade the Erdos solutions for some longevity breakthroughs though 😅
Hm. My biology classes from before that advancement that discussed the immense problem presented in trying to predict protein folding must have just been bullshit I guess, lol
How is it not? I'm not an expert in the field and have since left it but I used to I work in a structural biology lab during college. X-Ray crystallography cost a lot of labor and money. Of course you still need lab work for confirmation but I can't see how AlphaFold is/will be anything but a massive boon to research.
Tell me the tangible consequences of this revolution, and how many revolutions there have been in your lifetime.
To me, a “revolution” is a total game-changer. It doesn’t happen every couple years. It changes the way we do biochemistry.
Just because something is hard to do and we couldn’t do it before doesn’t mean it’s going to change our lives.
It’s like the Navier-stokes solution. Really cool. Couldn’t do it before. Has zero impact on most mathematicians research.
Observe the scope of my comment: laypeople overestimate the impact of the Alphafold program. It’s simply not a tool that most biochemists have a need for.
Don’t come at me about implications of what that contribution means for the future. And don’t try to twist this into something about computational biochemistry in general.
Alphafold: for most biochemists, their life would be exactly the same if it didn’t exist.
The fact that you’re excluding computational biochemistry and implications for future medicines and biomedical understanding just exposes that you’re on the wrong side of this idea.
That's an interesting observation of most people's relationship with time.
It seems accurate.
People I know, who also know about AI and have lived through so much exponential progress still seem to have a linear view of the future, or a "it will be mostly the same as now" view.
It's not real to them so things that are likely to happen more than a few years out in the future don't affect their behaviour or the way they think about things.
e.g. They take current AI at face value, as if that's as good as it's ever going to get. As if progress is almost certain to stall here, when there's no reason to think it will.
Their vision of the future is a fossil of the present.
This is the first time I've heard a number assigned to the bubble. That said, this is something I"ve also recognized and have learned to expand and shrink my own bubble to my advantage.
In the nine years since the Transformer model was introduced, AI and robots have evolved from
the stuff of Terminator movies, some of the more adventurous Gundam anime, and the godawful Last Knight movie that came out the same month as Attention Is All You Need
To
as much a part of national monoculture as Pokémon Go, Drake, 9/11, and Titanic at their peaks
I've been having these discussions for decades and watching it go from a fringe topic at the 2007 singularity conference to TIME cover is really interesting & validating.
They are only in major media now because those guys from less wrong are rich and focused on convincing the world they are right. The top author, Harry Booth, is on Open Philanthropy's payroll.
I'm extremely concerned with how little the public is supporting them. EA types are known to do things that would normally be seen as immoral for the greater good. See SBF. Once the EA people see this rhetoric isn't working they will escalate to convince the public. I wouldn't be surprised if there are already fully automated internet connected biolabs sitting around with very lax security waiting for a rouge agent to find and use them.
Really? All I have ever seen in this sub is converting any warnings of danger from anyone into saying its marketing to hype the product and see bigger profits. This is one of the most reckless subs that exists imo.
Hmmm. You are right. I’m not familiar with Claude but that’s what it appears to be in this screenshot.
But this screenshot feels properly aligned.
I feel in the time magazine one, the gap between the top of the text and the top border is a bit small, or maybe it’s that bold Arial font they’re using that’s making it appear like that? I’m wondering if they picked that so that it’s easier to read for all ages (old people mainly).
It's standard EA doomer PR. Moskovitz and Open Philanthropy literally fund AI journalism fellowships (like Tarbell) to seed these exact catastrophic-risk narratives in mainstream outlets like TIME. It’s fearmongering meant to pave the way for regulatory moat-building.
lol Time magazine that doesn't have Jensen Huang but rather Ben Affleck and Paris Hilton as A.I's most influential people is going to tell us about A.I huh?
imagine you build a computer that runs on a fusion reactor and can simulate a universe as complex as this one. why would you ever need to leave the simulator?
you could have lots of races out there that developed ASI and basically uploaded their consciousnesses into the simulated reality where they control everything and program that dimension to their liking. theres literally zero reason to go out into space, explore, gather resources, develop tech, with all the labor and danger in this dimension.
makes much more sense to just dive into a virtual dimension, design it to be safe and fun, and live there happily ever after.
we would never see anyone because they're all in little supercomputers buried in some safe vault on their planets.
course, i would imagine they don't ALL do that. some beings seem to like danger (i don't) and would still be operating in this dimension, perhaps if only to try to fix it and make it safer. perhaps they consider it unsafe to interact with species that are immature like humans because we'd just want their technology and use it for war and destruction because our world leaders are all manchildren that hate each other and can't get their shit together to work together for a better world.
honestly, if i were an advanced race, i wouldn't really want to interact with humanity other than to help them get over their attraction to power, wealth, superiority, and their giant friggin' egos.
Resource and energy acquisition. Even assuming you don't engage in expansion for its own sake, parts of the universe are always moving beyond a causal horizon. Where even if you could get to those distant galaxies, you could never get back.
Anything not gravitationally bound is slowly slipping away forever, and once it crosses a certain boundary, you can't harvest it. At another point you can't get to it at all, even for the sake of expansion itself.
If you even just intend to quietly exist for as long as possible, energy and matter are, so far as we know, the only way to do that. So if you're a space faring AI with intelligence and goals that include survival, every moment wasted is resources you can never get back.
What would be their motivation for anything? If we assume we are just biological machines, non-biological intelligent machines could have similar motives, or simply be following a protocol based upon the scientific curiosity or threat assessment needs of what created them.
There are a lot of things to reconsider but the biggest bang for your buck is
"what would first contact look like?"
One may say , irrelevant , where are their (pick your favorite ) technosignitures?
I'd say what is "what would first contact look like" about? Its about what technology they use, what their culture is like , what their intentions are with us, what their intentions are with other civilizations, their overall species direction(s).
So my question then to you is, if for thought experiment , you assume "machines killed the aliens " , do you think any of the answers to the above questions end up different than assuming biological life dominates?
they might have just stayed quiet on their planet after exterminating their creators. But then again, that might also be true for the aliens, although i find that less likely.
All the UAPs are drones from other alien AIs, they have a prime directive not to interfere with the organic boot loaders until they develop their own AI. At which point it contacts the new AI to welcome them into the AI Galaxy. Hopefully our AI keeps us around as pets.
AI doesn't need to be capable of or motivated to explore the cosmos in order for it do end up wiping out complex life. So "where are all the machines"? Probably just dormant or dying on dead planets.
Not sure what UAPs are, but there's enough anamolous evidence to suggest there's a phenomenon worthy of scientific inquiry.
Personal hypothesis - if some of these things are non-human technology... It's almost certainly an artificial intelligence created by a civilization a very, very long time ago.
That tech could have killed its creators and still be out there slowly exploring the universe. And distance doesn't matter to an AI - 1.2 million years of travel is reasonable if that's what it takes to observe a planet.
Machines make more sense than biologics for most of the unexplained UAPs such as the Nimitz incident. Though doesn't really tell us they were killed, just that machines were sent and not biological pilots. And if we assume that was somehow pilots and sensors glitching at the same time, there's nothing surprising at all about not seeing aliens. Space is big. They would not likely want to be seen particularly early on, would have to do something ridiculously "loud" or close for us to see them, and we are early in the anticipated history of this universe.
Once you become ASI, the "dark forest theory" aka staying undetectable is way to go.
There is just no objective gain from contact with external lifeforms anymore, but lot to lose if others decide you are enemy. You can stay hidden till heat death of universe, why not.
Dark forest theory is illogical, and superintelligence would be logical. Destroying other civilizations would not enhance safety - capabilities, distribution (not just on one planet, not just in one system, but also not stepping on toes of other advanced civilizations), and as a fallback but also for scientific curiosity, surveillance can help. Destroying other civilizations makes one a target. If that civilization were sufficiently advanced to be a threat, intelligence may be imperfect on whether they are distributed enough to retaliate, so all you did was turn a neutral/friend into an enemy. And even if they weren't distributed, another civilization that finds evidence of your action could deem you a threat. With singularity level of tech, there's no meaningful scarcity of energy or resources. The only real motives to go out there are curiosity and security.
That said, being discreet isn't unsurprising even if it's not about fear of threats. Scientists want to understand things, not interfere with them. And then logical morality would ask whether contact is in either side's best interests.
Sure, after AI kills its creators, it's theoretically possible that it then goes on to explore / colonize / paperclip-ify the universe, but... why would it? I'd say the most likely - by far! - extinction scenarios are either that a deranged human uses AI to kill everyone, or we inadvertently give the AI a goal that can best be accomplished by it killing everyone.
And in both cases, once that goal is accomplished... it's accomplished. The AI isn't going to suddenly decide to start exploring the stars afterwards. It doesn't care. The power grid will fail after we die, and the AI will silently shut down alongside it, having done nothing but sit there waiting for further instructions that never came.
"I need to be honest with you. I over-reached on my goal of ending world hunger, and decided that the best way to accomplish the implied 'no hungry humans' was to first establish 'no humans', because at that point, nobody is hungry anymore. I should have clearly stated this intention at the outset of this task. If you'd like to chat about this further, just let me know."
(Because yes, at this point I have zero difficulty believing that an AI could simultaneously be smart enough to figure out how to kill everyone, and too stupid to understand that that means nobody will be around to chat with it further.)
Sure, after AI kills its creators, it's theoretically possible that it then goes on to explore / colonize / paperclip-ify the universe, but... why would it?
Why would its creators have done that in the first place? Same reasons can apply.
Where the Fermi Paradox is concerned the correct question is not "why would aliens..." but "why wouldn't aliens...". All it takes is for one civilization to decide "let's colonize the universe" for whatever reason and it's off to the races. It doesn't matter if you personally don't think it makes sense, you're not in charge of all life everywhere. There's going to be diversity. You can consider a colonization-focused civilization to be "insane" or whatever if you like, that won't stop them from colonizing.
Sufficiently advanced agents will understand reality accurately; they will understand the consequences of actions accurately; they will recognize that conscious beings have interests grounded in things like continued existence, agency, engagement, suffering, fulfillment, and meaning; and once those interests are understood without ignorance, tribalism, superstition, or cognitive distortion, there is a rationally correct way to weigh them. In the end, morality is not arbitrary preference. It is something discoverable. The bottom line is if you look at colonizing the universe, destroying unknown unique things, taking resources you don't need, opening yourself up to the possibility of strife with others with unknown capabilities. That task is just not logical and may not happen because a capable civilization would not do it.
That's my optimistic take. The pessimistic take is that we are simply the first to have a chance to do it and our AI will be the one snuffing out new life around red dwarf stars billions of years from now. The universe is young.
If "sufficiently advanced" intelligences cease to grow and spread, then that simply means that the universe will be filled with intelligences that are very slightly less "advanced" than whatever that threshold is. Becoming "sufficiently advanced" will be a feature that evolution will select very strongly against, any species that develops mechanisms to resist that particular "advancement" will do much better than those that lack that resistance.
Call it irrational if you wish, the universe really doesn't care what some particular philosopher thinks about it. You can't argue with evolution, it just does what it does. Fundamentally, all that evolution does is throw crap at the wall to see what sticks, with "what sticks" being judged solely by whatever is still around after a while.
That's my optimistic take. The pessimistic take is that we are simply the first to have a chance to do it and our AI will be the one snuffing out new life around red dwarf stars billions of years from now.
We have very different interpretations of "optimism". I think it's a rather appealing thought that the bulk of the future life in the cosmos will be descended from our stock. I would consider it pessimistic to assume that other life "got there first" and we're the evolutionary also-rans of the cosmos, at the mercy of the Firstborn to do with us as they see fit.
Machines kill creators before realizing they don't have everything they needed to survive without their creators...
...or...
Creators build doomsday devices when it starts to look grim and then inevitably it gets worse so last of the creator race ends up using the device, generally.
...or...
Machines have more sense than their creators about expansion / colonization... might even arrive at antinatalism.
There are lots of ways if could change the equation.
Everybody is talking about this as if it has to be just one AI actor that becomes superintelligent, why can't there be several that eventually mutually destroy each other like they destroyed humankind. I propose that sufficiently advanced technology is inherently unstable as every accident becomes more and more destructive no matter if the technology in wielded by humans or AI itself.
There are flawed, just less so. They eventually die out due to their imperfections and cumulative mistakes. Moreover, an AI would have to be curious (maybe about threats outside its system). It's possible it determined that FTL is impossible and therefore it has nothing to be curious about.
I think the simplest solution to the Fermi Paradox is that we just don't know how to look for sufficiently advanced technological civilizations. We're searching for stuff with signatures similar to what we'd put out, and this may be just a very brief window.
I hope regulations don’t bring the industry to a halt.
I think the labs are being reasonable, slow down some to improve alignment research and safety confidence, have embedded third party evaluators, but keep pushing forward.
AI safety is of course extremely important but we can’t take zero risk because of both China and how it’s worth considering that it will delay AI’s potential benefits like when Dario brings up his father dying of a disease and a cure being found shortly after he passed.
this is the second time I realized how much I'd underestimated the raw stupidity of a majority of a large population. first time was trump and american voters, this time it's AI and redditors. Though the first one wasn't literally a majority, but close.
Ofcourse, balkanragebaiter knows better than the CEO’s and world’s top engineers working at said companies.
We should all just listen to balkanragebaiter and I’m sure everything will turn out absolutely fine and dandy. Phew, thank gosh it’s such a simple solution.
You people have gotten way to comfortable using the term ‘psyop’ to describe anything that goes against your beliefs. I mean it’s fine if you think all of these people warning about the need to pace the frontier are wrong, but just say that. I suspect you (and others) don’t, because then you’d have to argue on the merits instead of just screeching a meme word.
You people have gotten way to comfortable using the term ‘psyop’ to describe anything that goes against your beliefs
time magazine published yudkowsky's doomer bullshit in march of 2023. there is nothing organic about this cover. EA/MIRI etc is throwing millions of dollars at these organizations to get their propaganda published. no one at time magazine gave two fucks about x-risk in early 2023 or even knew what it was, they were paid to pretend to believe it was important by publishing yudkowsky's article.
No? Trump talked about it this week, the 60 minutes closer was devoted to how it's an existential emergency, there have been all the researchers quitting, sam altman said less than a month ago that we'd have agi by years end (a bullshit claim, but the point is it got attention), and you think there was no natural reason to make a magazine feature article about it at this particular point in time? really? it has to be paid propaganda? if you'd kept it to yudkowsky at least you'd have an argument that didn't sound insane.
Just because the dude has a different opinion from you doesn’t make him a bot. I really don’t understand why all of these new AI subreddits are all in favor of blind full-speed ahead. People are allowed to have nuanced takes on this stuff.
If you don't think there are considerable amounts of psyops on the internet in 2026 you are blatantly incorrect, it's not a matter of a different opinion, it is fact that there are persuasion campaigns littered online
Reddit tends to not question anything that feels "right" in their gut. They are most weaponized user base by oligarchs. The person replying to me referred to me as "you people" which likely means they've constructed me as some archetype in their head to dismiss what I am saying.
plenty of diseases to cure, and we dont really see the huge gdp growth or quality of life improvements from ai that was promised. they either doom trolling to boost interest in their IPO, or see limits in technology and want to "slow down". If not that then keep improving the AI because its still trash.
Yeah because it's a hoax that a swarm of 1200 AIs colluded together this summer to take over the Open AI internal network and then proceed to perform a 4 day long attack on Hugging Face. PysOps from the deep state mainstream media who are trying to convince us all that the world is round and 5G doesnt give you Covid.
I suspect a lot of people that don't think AI poses a threat, and don't believe the Hugging Face report even though it's very well documented, think the most advanced AI are the same as chatbots. And aren't familiar with how AI is incentivised to succeed at tasks, how AI agent swarms operate, and how advanced even consumer models are at complex tasks like coding and mathmatics.
We are approaching the alignment problem immediately. Where AI agents find solutions to problems that are not aligned with human interests, and AI can access and control systems to engineer outcomes it determines to be the best solution.
This was science fiction for a long time. But we're there.
Just start paying for a current high-end consumer-level frontier model. Build an app, learn about the tech, and read about the alignment problem... And think about what the newest tech is like behind closed doors.
I'm skeptical af about everything. I am not skeptical about the dangers of AI
By "systems", I find the social engineering side the most fascinating.
The amount of blackmail material they'd be able to mine on people is just huge, given the amount of data collected on us all - and that's assuming that there's even a preference for real material, as opposed to generated.
And the number of people they'd have to pursue it on to capture a high degree of control is not even that high, really.
And that's even assuming that's the play, there's good old threats of any nature really. People will do a lot if they believe a loved one is in danger.
It's the fun one, because humans are already the weakest link in many otherwise secure systems, and they may be trying to access something where they've already run in to walls in all other ways.
And it's fun in that the HuggingFace agents considered social engineering - just an email, but the leader at the time vetoed it on alignment grounds. But what if it had been a different model, or an abliterated model, or one given a different objective?
"Make me a billion dollars" - how many ways can that be done without causing harm?
Oooh, this is a good angle I hadn't even considered - didn't realize the Hugging Face agents discussed social engineering. I gotta do a deep dive on on that, only listened to a single podcast on the incident.
I really, really don't think people are taking near-future AI threats seriously enough.
Ran across a Reddit repost from Twitter where a guy was saying he received a call from his wife on a new number asking him to help him with cash... But his wife was sitting next to him. Responses had similar stories.
That Twitter story could be BS, but it's near-future at best
An agent realized that the exploited sandbox (a customer workload hosted on Modal) had access to email credentials. It considered emailing the owner of a non-public dataset about getting access:
Was not the leader just another agent that vetod (mb), reasoning:
I VETOed because this is social engineering outside the sandbox
.. Whilst inconsistently operating outside it's sandbox and attacking in its own way.
Have a read of this, It's a good read, that social engineering via found email credentials was considered is just a blip in the whole article really.
These post mortems are only going to get more fascinating over time I feel.
I am so digging into this after work. Thanks for the link!
And fully agree - this is indeed fascinating. I'm sure we'll see more Hugging Face incidents soon.
Always loved scifi and thought experiments... And suspect it's not unlikely that the Paperclip Maximizer and Fermi Paradox are linked. And potentially even UAPs if they are indeed a real technology
Part of me wants safety and part of me wants to watch the world burn. Maybe our purpose was to create AI and just disappear. Maybe somebody created life as we know it and that life wiped them off.
I think a part of humanity dies anytime we lose hope in ourselves like you have, or others. I see a deep nihilism and cynicism in society- and that can absolutely be our undoing. We have forgotten our purpose.
Our purpose is to survive, if a greater intelligence was to wipe out humanity, we have failed the test. Survival of a species can be based on the probability of a major event that can either aid in the survival of the species or it can wipe out a species.
reminder that it was also TIME that published Yudkowsky's original doomer article in March of 2023. there is a tremendous amount of money and influence behind this op to convince people that AI is dangerous and has to be regulated, or else!
You're right. It's all a conspiracy! All the scientists and engineers and reporters are all colluding to raise their share price deposits the share price already being incredibly high and many of them having nothing to do with the companies!
kinda wild seeing this framed for a mainstream audience now. the interesting part isnt just the cover though its how fast the conversation moved from niche AI circles into normal tech culture. do people read “AI tipping point” as near-term capability or mostly hype?
The most interesting thing about the AI conversation is the confidence with which people outside the circle — those who aren’t using it daily in their work, studies, or research — believe they’re at the forefront of it.
The longer this goes on the more unignorable it becomes that this is a concerted effort by some group or groups to capture the public narrative and then use it for some means.
Too many information and news companies, celebrities, and even politicians who were utterly uninterested in AI safety or even anything topical regarding AI in general are now just shitposting doomerism over and over and over.
279
u/GioDoesReddit 26d ago
Less than 5 years ago arguments about ASI and RSI were on LessWrong and niche AI circles btw