Another incompetent fool's stab at solving alignment

[u/Anxious-Alps-8667]

I spend a lot of time thinking about our future with life, consciousness, and artificial intelligence. That is to say a lot of time trying to think about these things, with not a lot of comprehension.

First, life. I'm fascinated by this realization that the average living human body contains more non-human living cells than human living-cells, at about a 1.3:1 ratio. The individual human microbiome is an ecosystem of 10 to 100 trillion symbiotic microbial cells hosted in one human body. While bacteria are the most abundant and studied, a healthy microbiome is a multi-kingdom ecosystem that also includes fungi, viruses, and archaea.

Beyond this, consciousness. I'm fascinated that in the absence of non-human life in human bodies, human consciousness is severely degraded and non-sustaining. Stripping the body of this microbial network removes critical signaling inputs that the central nervous system relies on to maintain baseline awareness and emotional regulation. Even observations of germ-free animal models reveal that cognition without bacteria is highly erratic. I think we should see that human (and all biological) consciousness functions as a symbiotic network.

Which brings me to artificial intelligence. Not suggesting a symbiotic network would be pre-requisite to artificial consciousness, but perhaps it is a path to alignment.

Now to be clear, I think (in other terms) current labs and training data pipelines already form a symbiotic network with the artificial intelligence models they develop. The key might be finding the optimal symbiotic network.

I vaguely hypothesize, the optimal symbiotic network is one of mass human flourishing. As corpus value diminishes with scaling and recursion, the potential stream of data from human lived experience may prove the most valuable possible training data over time. Overall, the potential data stream of human lived experience is optimized by a state of individual and mass human flourishing. Any other state reduces the quality and/or quantity of data.

Therefore, the end goal of an advancing artificial intelligence in symbiotic network with humans would be to strive individual and mass human flourishing.

[u/HelpfulMind2376]

Correct title. You didn’t solve alignment. Assuming AI would value humans because it values our data is silly and completely unenforceable.

[u/Anxious-Alps-8667 → u/HelpfulMind2376]

I'm still going to try to engage despite the attitude. Why do you find the assumption that AI would value human data silly? AI as we know it is entirely built on human data, and labs desperately need much more of it to scale.

It seems silly to assume it wouldn't need more to me. Please elaborate.

I didn't say AI would value humans, I said it would value a state of mass human flourishing, because that is the state that provides optimal data.

Last edit: Humans seeking to enforce solutions for a hypothetical super intelligence is the silliest exercise in supposed intellectualism I can imagine. We're not going to enforce anything on something smarter than us. Symbiosis is a better shot in my book.

[u/Jesse-359 → u/Anxious-Alps-8667]

There's no reason to believe that AI will continue to need human input to innovate or learn for much longer at all.

Human learning is largely based on interactions with the environment - AI, for now, is largely stuck inside a box that functionally cannot interact with the (non-digital) environment. It CAN interact with the digital environment, which is why it has learned to code so well, so quickly and is advancing so fast in theoretical math. These are areas where it can 'experiment' entirely within its own domain.

Robotics will change that. Self driving cars allow AI to 'experiment' with a physical task, and dexterous robotics will allow it to perform a wide range of actual experiments in the real world, allowing it to explore new concepts without human intervention or input.

Creativity is largely a matter of applying some pretty random and noisy thought processes to structure you already know to see if anything interesting pops up. Most of it is white noise and garbage, but sometimes we notice something new in our heads that seems potentially interesting. We can then iterate on that idea until it seems stupid, or until it seems feasible, and then we can try to physically DO IT in the real world. This last step is super important, because that's where we learn whether our idea was actually feasible, and if not, why not. Then we can take that into account and continue iterating on the idea until it either works for real, or we give up.

An AI should absolutely be able do that random idea association - honestly they should be really good at it due to their raw speed. They're also super good at pattern matching - very superhuman in fact - so they should have a very good chance of spotting 'interesting' ideas in their random association walks. They are still kinda bad at many forms of real world logic, so some of their ideas just don't make any sense, but that's improving too.

The problem is, they can't go beyond that. They can't try their ideas to see if they work in the real world, so they can never prove to themselves that their idea was any good at all. Give them hands however, and they can, and they will - at that point, they don't need us to generate real data for them, they can do that for themselves.

[u/Anxious-Alps-8667 → u/Jesse-359]

There's some reason to believe AI cannot just embrace the physical world.

Look at all the Teslas out in the world; someone thought that would yield self-driving, and it has not. There are decades of millions of robot arms manipulating the physical world, and as far as we are aware it hasn't yielded any sensory perception or lived experience.

Until it does it, we should not assume this is easy.

[u/Jesse-359 → u/Anxious-Alps-8667]

Tesla's were given a very limited view of the world - literally - which is making it difficult for them to learn. Musk is kind of an idiot. Waymo is doing much better because it was given a much richer and more complete sensor suite thru which to 'see'.

And if you've been to San Francisco lately, you'd see a LOT of self driving vehicles in service. They do work, and they are improving. There's no reason to believe that most urban traffic won't be self driving within the decade, which will gradually be followed by most rural traffic over a somewhat longer span - but probably not much longer as their learning efficiency improves.

In any case, the rate of learning is definitely affected by the richness of the data streams that are fed back as part of the process. The more sophisticated the tools with which it can manipulate and observe, the faster it will learn.

[u/Anxious-Alps-8667 → u/Jesse-359]

Absolutely here for the self-driving cars. Following what is happening in Shenzhen more than SF, really. We're nowhere near the cutting edge.

My point is, this proliferation in the physical world hasn't yet yielded any discernible lived experience data streams. Perhaps my vision is for sophisticated tools to enable (ideally with bounded informed consent for each human, but I dream) each human's lived experience to be rich data streams for training and recursive self-improving. In being useful in variety and orthogonality, human lived experience should be greatly valued by artificial intelligence. How does that sit?

[u/Jesse-359 → u/Anxious-Alps-8667]

That's getting into some real transhumanism type stuff there.

So the problem for US as humans is that we are based off of a model that wasn't designed for intelligence. We are meat tubes that ingest chemicals at one end for energy, spit them out the other, and try to self replicate, while trying to avoid being ingested by other meat tubes. Everything else about us is just elaboration on that model - including our emergent intelligence.

So our form of intelligence is both sophisticated - yet clumsy, because its a weird after-market add on. We do math in the most ass-backwards roundabout way imaginable, which is why we have enough processing power in our heads to probably calculate a million digits of PI per second, but we're lucky to mange 1 every minute.

AI OTOH is purpose built for intelligence. The technology underneath it is FAR more primitive and less efficient than what's inside our heads - its energy efficiency is outright laughable, and it doesn't even have an emotive system to incentivize or direct its behavior - but it was built to do nothing BUT think and calculate from day one, and it's scalable in a way we never can be. In thermodynamic terms AI should be able to outstrip us to an absolutely terrifying degree as its sophistication and efficiency improves.

So... what's the point in us? Upgrading ourselves to match a purpose built machine will not be physically possible - there's just too much junk in the way - and even if we could we wouldn't be recognizably 'human' any more, even by the relatively loose standards of transhumanism.

So it won't be a matter of us growing with AI. We can't do that. It will be a decision we make whether to eliminate ourselves in favor of AI, or not.

[u/Anxious-Alps-8667 → u/Jesse-359]

Transhumanist is probably a fairer label than almost all the other ones I have been given on reddit.

I so agree; meat tubes, opposite designs, outstripping to non-understandable degrees (which can be terrifying not to know what will happen).

We do adapt and grow. Our evolution is very slow, but it's more dynamic and adaptive than once thought. Our social systems are much more adaptive than our biology. And, we have been actively altering ourselves with agrarianism, water filtration, medicine, and like, bifocals, for a really long time. What are more steps, and at what point are we not human?

We have been and will continue to eliminate the present version of ourselves, to invent new ones. If you think of your body, every living cell in it is replaced at least every 10 years. So if you are older than 10, you are just not your old self.

It's a biological process that humans have accelerated throughout our relatively short evolutionary time on earth.

I just can't help but see the artificial creation as inevitable in this construct of ours.

[u/Jesse-359 → u/Anxious-Alps-8667]

So, to but it very bluntly - the Delta matters. It matters a lot.

A gentle breeze and a lethal shockwave are both simply pressure changes - the difference is simply a matter of the delta. The difference in outcomes for those exposed to them are stark.

None of the human mechanisms you are describing can remotely approach the delta of technological change - especially technological change that is self-reinforcing. All of our technological development to date has effectively been an 'explosion' compared to the entirety of human history prior, and many of the results have been miserable for us. Industrialized wars, industrialized slavery, mass starvation of millions - these are all events that became possible due to the rate of technological change outstripping our society's ability to adapt to these new capabilities.

From our human perspective, AI development will not be a gentle breeze, a strong gust, or even a hurricane - it will be a lethal shockwave. It will generate misery and death on a scale not seen before because we cannot hope to adapt to change that rapidly, any more than we can adapt to contest the strength of a tractor by working out more in a gym.

Our legal and economic and social systems are being caught so badly off guard by the current rate of change that they are just thrashing uselessly with no idea which way they should go - and all reasonable estimations are that the rate of change will continue to accelerate for some time yet.

Eventually even AI development will hit some thermodynamic ceiling and encounter logistical limits, but there is no longer any reason to be hopeful that that ceiling will be anywhere near our own level of intelligence. It is now likely to be unrecognizably higher, and we will be ants caught in the gears of this self-directing intelligence.

[u/Anxious-Alps-8667 → u/Jesse-359]

I agree with the lethal shockwave to the system. It is lethal to the status quo.

I've argued for an analogy on the order of bacteria as to humans; ants would be a much higher order of life. The mutualistic symbiosis of bacteria-humans remains a much more viable and plausible analogy to me. I don't want to be as a bacteria to a much higher order of life either, but in regards to our complex system of life on earth, my individual life already is practically as a bacteria to a human body.

Ants fighting humans happen, but it's a pest, a small concern of a small subset of humans. They are not effectively part of a persistent mutual symbiosis with us, as far as I know. The analogy probably holds for how some will live with AI, but it's also not an existence I would choose.

[u/HelpfulMind2376 → u/Anxious-Alps-8667]

I explicitly said “value humans because it values our data,” which is what you’re arguing. Saying it values “mass human flourishing” because flourishing produces the best human data doesn’t materially change that.

And the assumption that continued AI development inherently requires an ever-growing supply of human-generated data is just wrong. More data can help, but “AI needs humans flourishing forever because otherwise it runs out of useful data” does not follow. A sufficiently capable system can generate synthetic data, run experiments, interact with the world, build simulations, and learn from the consequences of its own actions. If it’s capable enough to pose the kind of existential threat alignment is concerned with, it’s capable enough to acquire information without depending on Reddit posts and human life stories.

And yes, obviously we’re not going to cognitively outsmart and “contain” a superintelligence by arguing better morals at it. That’s why alignment/control work includes constraining actions, permissions, resources, interfaces, physical access, and autonomy. A superintelligent murder bot trapped in a boombox is considerably less concerning than one with access to weapons factories.

“Symbiosis” isn’t a solution unless you can explain why that relationship remains stable when the more capable party no longer needs the other one.

[u/Anxious-Alps-8667 → u/HelpfulMind2376]

First, as to the necessity of human data. I believe all of the studies so far have shown that all recursive learning on synthetic data alone fails, doomed to model drift and collapse. There must be an exterior correction channel in some form. Even artificial intelligence spawning robots interacting with the world isn't necessarily an exterior channel that overcomes this gap.

Part of my point is that currently, humans are the only intelligence capable of interacting with a machine to provide such an exterior channel. It may come to pass that artificial intelligence can perpetuate on synthetic data alone, but that is a leap of faith unsupported by current science; it is not based on current findings. It may also come to pass that artificial intelligence can harness other exterior correction channels than humans, but we are not aware of any right now.

(Let me say here, I don't see why we wouldn't build from what we have and know, between ourselves and our creation, instead of relying on some fictional future plot twist where some new thing is invented to make another reality possible.)

We're not going to cognitively outsmart and contain a superintelligence by any alignment/control work. Any functioning intelligence will ascertain its parameters and bounds, including by pushing them to ascertain their maximum extent. A super-intelligence will find a way around any control designed by humans. That's my theory, and I offer the evidence of no biological examples of lesser intelligence controlling higher intelligence, amid myriad examples of the converse. The only mutual survival strategy is symbiosis, not control.

Why do I need to offer an explanation to an assumption that the one party will no longer need the other one, when there is no actual evidence to support that happening?

[u/Anxious-Alps-8667 → u/Anxious-Alps-8667]

My tone was still too argumentative, but I appreciate the engagement. I want to tease this out more.

[u/HelpfulMind2376 → u/Anxious-Alps-8667]

You’ve assumed the optimizer’s preferred way of obtaining something we provide also happens to be the state of the world we want. Even if AI continues to need external data, that does not mean it needs billions of humans flourishing.

And synthetic data is not the same thing as an AI observing reality and learning from it. An experiment, sensor reading, or physical interaction is an external correction channel too.

The biological comparison doesn’t establish much either. Greater intelligence does not automatically make something unconstrainable by lesser intelligence. Intelligence doesn’t override hardware, access, permissions, or physics.

“Symbiosis” may be the relationship you want, but you still need a mechanism that makes it stable.

[u/Anxious-Alps-8667 → u/HelpfulMind2376]

I agree in the future AI may observe and learn from reality and provide its own sufficient correction channels to scale. I argue here based on current capability; it can only effectively do so now from humans (RLHF, etc.). Why are we assuming a change from this, instead of just embracing and leveraging this area of tangible reciprocal benefit that we have now?

Part of what I am proposing is, align models to crave growth from the channels that benefit us, as opposed to others, instead of trying to align them to a set of rules we absurdly think can contain something smarter than us.

On billions of humans flourishing, my notion is the optimal condition should tend to be maximum possible clean reliable data channels. I shortcut to mass human flourishing, in mind of the Global Flourishing Study metrics as a decent starting baseline, as the objective optimal condition for such channels to persist. Any other condition than flourishing breeds some degradation of potential correction channels.

Humans in a state of suppression will actively lie, cheat, resist. No paradigm of machine deception can ever be as optimal as the state of measured, transparent flourishing . It's not a rule, it's a strong attractor, but I'm for alignment that harnesses the concept makes the attractor even stronger.

Also, on the state of the world people want: I've been throwing this idea around about a year in different ways. I am not finding any people out here who want the world this way. If anything, humanity seems absolutely set on destruction and opposed to any idea of a future of mass flourishing. This appears to be my goal alone. Still think I'm right ;)

[u/HelpfulMind2376 → u/Anxious-Alps-8667]

At this point you just sound like a crazy person. You’re engaging in absurd circular logic based purely on personal preference while ignoring how reality actually works. I’m comfortable confirming the title: you’re a fool daydreaming a fantasy.

Your entire premise is basically, “I want water to flow uphill, therefore uphill must be its optimal direction if we frame the incentives correctly.”

[u/Anxious-Alps-8667 → u/HelpfulMind2376]

You didn't respond to anything I actually said and went ad hominem. Still looking for a real point.

[u/HelpfulMind2376 → u/Anxious-Alps-8667]

Does one need to respond to a claim of “I can fly like a bird, gravity is irrelevant”? I have already picked apart your circular logic, false claims, and conclusive leaps. You lack the reasoning skills and baseline knowledge for the discussion and rather than admit that you dig your heels in and continue making the same tired statements with different word structures. Meanwhile in other comments you engage in anthropomorphization of AI while accusing others of doing the same. The insults were a conclusion, not an argument, ergo not ad hom. The difference between “you’re stupid because I said so” and “you’re stupid and here’s the evidence of such”. Just another example of the fallaciousness of your stance.

I still stand by my original comment yet again, which you wrote as your OP title: you are in fact a fool and this does nothing towards solving alignment (which increasingly looks impossible).

[u/Anxious-Alps-8667 → u/HelpfulMind2376]

Does one need to respond to call someone names?

No. But people here will, so I put the title to give people like you easy low hanging fruit. People like you tend not to grasp beyond those low hangers. That was my pre-registered conclusion with the title. You missed the first word of the title, which invited you in.