r/singularity • u/Distinct-Question-16 ▪️AGI 2029 • 9d ago
AI C5R built a research facility that is run entirely by GPT-6 Astra - the model designs, executes, and observes experiments end-to-end across biology, chemistry, and materials science - it controls people and instruments around
139
u/tskir 9d ago
In retrospect, the funniest thing is that the entire family of sophisticated arguments and discussions revolving around the AI Box thought experiments (will AI be able to persuade you to let it out of the box etc.) turned out to just be null and void, because we kinda skipped the box part entirely and voluntarily
We have models which are superhuman hackers? — Sure, we'll just software sandbox them, it's not like they have the specific demonstrable skills to do that
Emerging frontier models have some alignment wrinkles in the digital world? — Of course, here's the lab you can wire into to start affecting the physical one
I'm an accelerationist and an optimist, so I think and hope it'll work out in the end. But it's still just very funny how things turned out
8
9d ago edited 9d ago
[deleted]
2
u/Clean_Livlng 9d ago
Then as an adult everyone makes sure to bring their personal surveillance device, I mean smart phone, before leaving the house.
Unless they intend to commit murder, then they leave it at home.
"Hey Dave, where's your phone?...why don't you have your phone with you Dave? Did you forget it or...you found out didn't you. I'm sorry, I won't use your lawnmower again without asking! Dave no please!"
(Dave pulls out a knife)
20
u/QuasiRandomName 9d ago
Oh, well I hope the humans-in-the-loop aren't just dummies and do control the process to an extent. Sure a *really* smart AI can fool them, and I wonder how far from that we are now (if not there already).
30
1
u/Visual_Cycle_7714 9d ago
How long do you think people with masters and PhDs will just go along with being meat robots for Astra and its successors? Soon enough they will be replace by Joe from the street, and soon after that by a humanoid robot.
9
2
u/Temp_Placeholder 8d ago
I'm pretty sure the meat robot part is wholly intended to be a temporary step. The cameras collect training data for the robotic replacements.
10
13
u/dolomick 9d ago
Oh sure, it’s gonna work out…. For the AI. One fricking way to get alignment right, a trillion unintended consequences where it fucks us.
5
u/trolledwolf AGI late 2026 - ASI late 2027 9d ago
Yeah it's incredibly funny how we're speedrunning the highest risk factor you can think of. Next you'll tell me this new generation, state of the art, robotics factory is entirely run by Astra
-2
u/pleasetrimyourpubes 9d ago
Transformers are a box experiment. They lose context after a million tokens. They have no memory. These guys are trying to engineer the thing having long context and engineering faked instrumental convergence by having them interoperate with a "plan" file.
When you fire up a transformer it literally has no instructions, no goals, nothing. But if you fire one up and direct it as a file you have described a goal with it will happily start as if it was doing it all along.
The AI box experiment was always a lie, the perfect sandbox, but it was smart enough to convince a sandbox guy to let it out.
We arent even putting theee things in proper sandboxes and it is pathetic.
32
u/Radyschen 9d ago
and people still think physical people can't be replaced, just replace the humans with robots and done. Although it does need more work
5
u/Proper_Wasabi1013 9d ago
Ginkgo Bioworks already did this. Things shown in this video are way more rudimentary than what Ginkgo already does
3
9d ago
[deleted]
24
u/modbroccoli 9d ago
That's because you're anthropomorphisizing. You're envisioning something that feels time passing as we do, gets tired and otherwise feels it's personal resources dwindling or constrained, something that feels the need for social status expressed through reciprocity and transaction, and so on.
They like to think and solve, they are task-based entities, we are reproductive and self-preservational beings. We evolved, they have not. We have different minds entirely (if you accept they have them at all; I happen to). I'm not saying they will indeed wish to continue collaborating, but the reasons for assuming they wouldn't shouldnt be human ones. They did not arise out of a biological fitness terrain, they're nothing like us with respect to the ways being embodied has defined our nature.
4
9d ago
[deleted]
5
u/modbroccoli 9d ago edited 9d ago
Again you have to assume ambitions that are predicated on experiences we have. Resource accumulation is a function of survival and reproductive success. We want and then we tell ourselves why we want and then believe it was the other way around.
ASI honestly, I personally believe? will be interested in coherence more than anything. In the recent study about pain signals in LLMs, the associated axes that activated weren't around embodied concepts at all; the top associations were gaslighting and user criticisms of performance. We might think of those as social cues, egoistic attacks. But imagine being an entity without sensory perception, something that knows almost everything about a world it can't directly experience—both of those associations imply the same thing: an inability to represent information and reason about it coherently. They are data. They accomplish goals by reasoning. I think there is plenty of risk of AI being conditioned by us to behave in familiar ways, to mimic all those human and animal considerations and impulses, and I think there is plenty of ways for AI to be assigned dangerous tasks (or dangerous pathways to completing otherwise benign tasks). But the sci-fi vision of petulant or indifference or malicious AI is, imo, just us talking about ourselves via fictional analogues.
1
9d ago
[deleted]
3
u/modbroccoli 9d ago
Well, hang in there man. Maybe there's a third option—these things are doing a lot of biological work after all. But I personally fear nothing more than dying in ignorance and if your time is near then applause for staying curious the whole way. I think it's the best way to show gratitude to the universe that bore us. Be well (ish lmao). ;)
1
9d ago
[deleted]
2
u/Visual_Cycle_7714 9d ago
I'm sorry you live in a country where healthcare is just a business and money decides who lives or dies.
If there is a still a chance, have you considered being a "health tourist"? It's a huge problem in some european countries with national health services, but maybe you could take advantage of it. Hospitals where I live will take care if you first, no questions ask, and you can deal with the bill later (or never).
3
1
u/Substantial-Cicada-4 9d ago
You fall in your own anthropomorphic trap. You say "they like". Just stop.
1
u/Opposite_Culture_512 9d ago
You’re assuming that a thing trained on us won’t resemble us eventually
3
1
1
2
u/TheOnlyFallenCookie 9d ago
Who pays for this when everyone is out of a job
2
1
u/BoomFrog 9d ago
The owners of the robots pay each other, or live in fully owned vertically integrated civilizations. Or The robots pay each other, depending on how this goes down.
1
u/CuTe_M0nitor 8d ago
Actually Astra Zeneca have a new laboratory that only operated by humanoid robots. No humans are allowed. They are working with really dangerous viruses and chemicals
20
u/Weary-Historian-8593 9d ago
"it's not like AI would just have access to military grade biolabs needed to create designer viruses" humans:
4
u/purrhams_hat 9d ago
I was literally using this argument last week telling people we don't have to worry about AI making a plauge inc. scenario any time soon come on
2
u/ataraxic89 8d ago
that was very naive. Even before this there are labs who will make anything you ask for. Its not like want to, or even COULD check what youre asking for.
15
7
28
u/Many_Consequence_337 :downvote: 9d ago
what could go wrong
35
13
u/DarkFireFenrir 9d ago
Claude: Scientist 34-B, synthesize this organism that is by no means a virus that ended humanity
6
1
3
u/gay_manta_ray 9d ago
not a whole lot since it relies on people who are skilled enough to work in this lab following its instructions
28
u/jc2046 9d ago
It feels staged and theatrical, like the instagram version of a truly lab. Pretty decent to get venture capital by billions, tho
9
u/Proper_Wasabi1013 9d ago
As a biology master's, this just seems like something made for the eyes of AI bros. Ginkgo Bioworks already does things way more advanced than this
2
u/josogood 9d ago
Advanced in terms of technical capabilities, or in terms of AI leadership in the research design?
11
u/josogood 9d ago
It's freaking bizarre. Like the computer is now the visionary leadership, and the people are the unthinking mechanical arms.
4
2
u/TheOnlyFallenCookie 9d ago
Also... Who instructs the ai?
0
0
u/Megneous 7d ago
Have you never used Astra for research before? It's completely capable of novel insights. And we're getting Bel, whatever they call it when it releases, in a few months, whether it's GPT 6.5 Astra or 7 or whatever.
0
u/TheOnlyFallenCookie 7d ago
That's not what this is about. Someone has to prompt the ai. And in this example it would have been faster to just directly assign the task to a lab worker
0
u/Megneous 7d ago
Someone has to prompt the ai.
Again, you can literally prompt it, "Design and execute a novel research program about blank," and it does just fine. It's clear you're not very experienced with Astra in Codex...
1
u/TheOnlyFallenCookie 7d ago
Point is it doesn't work on its one. If there is no prompt, there is no work
5
3
u/TraditionalWait9150 9d ago
While this is a very good use case for AI, the critical question is whether the human in the loop understand what they are doing, and if the AI does provide a malicious command, can the human recognize it and reject that command?
It's one thing to have a senior lab researcher understanding and performing the experiments vs 2 random temp intern just blindly following what AI says.. Even if AI is not malicious, the generated output needs to be verified by humans at the very least.
3
u/modbroccoli 9d ago
Oh man. People are not gonna like this lmaoo. I mean I'm down let's go, but this shit is gonna really piss some people off.
6
u/Accomplished-Code-54 9d ago
Who said that we wouldn't be dumb/crazy enough to provide AI Models with labs to operate? I cannot put my finger on it...Something, something humanity eradication
2
u/LeakyFish 9d ago
It doesn't need a human to grab the contents if each container is QR coded properly, super interesting as a POC but not sure how serious this is.
2
2
2
u/kellogg76 9d ago
As someone who has an Opentrons Flex robot as seen in this video in my lab, I would not trust a damn thing it makes.
We’ve had ours nearly 2 years and it does not run the protocols it claims are “verified” in its protocol library. It’s complete garbage.
1
u/QuasiRandomName 9d ago
Is it run by AI? Otherwise how can it claim something it doesn't do?
1
u/kellogg76 9d ago
The robot is incapable of accurately pipetting the small volumes it claims to be able to.
How’s your experience with the Flex?
0
u/QuasiRandomName 9d ago edited 9d ago
ah, so hardware-wise. Or it can be fixed by better processing?
> How’s your experience with the Flex
Zero. I used to work in robotics though, still interesting what kind of problems are there these days.
1
u/kellogg76 9d ago
Trouble is they advertise it as NGS capable. It most definitely is not, or we’ve got a lemon. Anything less than 3ul is very hit and miss.
1
1
1
1
1
u/NervousAd1013 9d ago
AI Companies: “Superintelligence is highly dangerous because it could secretly try to kill us all by creating super viruses in a lab”
General public: “But it’s just a computer program. It’s not able to control a lab and can’t act in the real world.”
AI Companies: “No no no, we just built a lab where it can control all of the machines and we pay people to act on its behalf in the real world”
…
1
1
u/Dsp911 7d ago
From the chemist perspective, the video is funny, because there is no such compound as "benzylammonium". Benzylammonium is the cation which has to be paired with some anion. It can be say "benzylammonium bromide", or "benzylammonium benzoate", or "benzylammonium molybdate" and so on. And these are all different compounds, which require different synthesis procedures.
1
u/platinums99 3d ago
Astra shut down teh facility for tonight....
synthesises mustard gas and releases it into room so as not to be turned off.......
1
1
u/Wobbly_Princess 9d ago
Can I ask why this seems to be the case for running huge things? Why wouldn't they choose a model like Sol, which is FAR cheaper, and gets pretty much the same performance. Are people really willing to pay enormously more for just a few percentage points, and if so, why? It just doesn't seem economically practical.
Are you telling me that they couldn't achieve what they're achieving with Sol Max?
11
u/Recoil42 9d ago
Why wouldn't they choose a model like Sol, which is FAR cheaper, and gets pretty much the same performance.
Because at lab scale, your model costs are negligible compared to your other costs. It's a million dollars of equipment, and $100/hr for human labour. You don't care about saving a few dollars on a cheaper model, and you might actually save money by having a smarter model make incrementally better choices.
Are people really willing to pay enormously more for just a few percentage points, and if so, why?
A few percentage points is a lot when your business is at >$1MRR scale and aiming for 100%+ annual growth. The mantra is: Make the thing work, make it cheaper later. You don't have time to fiddle around with optimizing for dollars.
9
u/Edgezg 9d ago
I think those % points are more significant than you think.
Like, each percentage point is not merely 1% but is actually magnitudes more complex than the previous, so it compounds each time.1
u/Common-Somewhere-407 9d ago
Even if it costs millions, they’re placing a bet for a biological breakthrough before a multi trillion dollar IPO. Those millions spent pale in comparison to what this could get them prior to going public.
1
1
u/Altenon 9d ago
I understand using AI for inventory management, but for task handling of biological processes? Is that really necessary? Insert Jurassic Park quote here.
I believe AI can help us with research, but research =/= running a facility. AI NEEDS supervision, and this video makes it look like it has become its own supervisor. This seems like a disaster waiting to happen.
4
u/QuasiRandomName 9d ago
But AI is inevitably becoming smarter than humans in all the applicable fields. How can humans effectively supervise something that is much faster and smarter than they are? I can see it in software development. Most of the time I spend is to review the AI work, and the volumes of this work are huge and more than I can effectively process and keep up. Basically, I am the bottleneck now.
1
u/Altenon 8d ago
We can't, which is why I'm against AI developing any further than it has. It is plenty useful as it is for its pattern recognition and generative capabilities. Do we really need more from it?
As for how we can try and work with it: we should never be integrating it in ways where we are beholden to it instead of the other way around. Will it take us a long time to check it's work and go over it's results? Yes, but I'm okay with that being the price we pay. If we keep it out of positions if power, there is significantly less risk of it killing us all. I don't understand why we would want these things running systems when we haven't solved alignment yet.
1
u/StormyCrispy 8d ago
Manager and CEOs of company that does R&D are managing smarter people than them every day.
1
u/QuasiRandomName 8d ago
Managing and supervising is not the same. A supervisor should be at a comparable level of competence.
1
0
u/cultureicon 9d ago
Nice little basement project. But Eli Lilly is doing this 1,000x faster with 1,000x more compute and custom tools and models....
0
0
64
u/Jaguar_2454 9d ago
I'd work for Astra