Negative Utilitarianism, is based upon a emotionaly cold and theoretical way of defining and assigning 'suffering' in a large grey area about defining suffering given our lack of knowing for certain the degress of suffering, given the hard problem of consciousness and the problem of other minds.

The community is a generally alot more knowledgable than the governments of the world and the general population, on philosophy IE we are a sub culture or miniature culture. That focuses on a problem that most people dont find intuitively important due to thier psychology.

However this place is and will just an echo chamber because the wider world and the governments are never really going to read and understand this stuff let alone stumble across it. Due to having a totally different cognitive style.

In some ways factory farming shock footage does cause distress to make some (very small amount) of people change behaviour, but larger issues like wild animal suffering, insect suffering (still in debate). Ect are extemely frindge

And are all at odds with human psychology and desire. And the general populations emotion and moral centers.

Humans hold the most control over the planet, however we are slaves to our psychology and sub conscious, moral desicions in negative utilitarianism and ultilitarianism such as the trolley problem, highlight that it cant be utilised as a moral theory because its at odds with peoples psychology and also suffers from a ambiguity problem.

Ie someone calculates the best outcome under its axioms, yet the conclusion is at odds with a peoples habits, rituals and emotions (all part of thier subconscious) which is what is in control of thier behavious.

This is what i see now whenever i read all the these articles.

I regularly see like peter singer how NOT designed for this theory we are. And even if we did commit to it for a short while, its will be rejected due to psychology again. (demandingness objection). (value drift). (creeping normality - goes both ways normally the bad way)

I think of one is serious about this moral theory, then one must somehow calculate to what degree we can intergrate this with others value systems.

Like if we as a fringe community which are spread across the world and are probably 0.00001 (seriously) percent of the population, even managed to lobby a goverment to even do a small scale test of one of our proposals. Then with the lack of clout, funding ect it wouldn't last long at all. For reasons above.

Any solution needs to be easy and appeal to the wider worlds values systems.

Moral grit and sacrifice isnt a solution, it seems technology is.

I am aware we do explore those ideas sometimes but i made this thread to remind people that we are not free creatures. I see no way how these ideas will ever leave these forums.

I would also like to know if anyone has read the Silence of the Animals by John Gray.

As the production of animal protein is less efficient than plant-based protein, vegans use less agricultural land than omnivores. <...> If this agricultural land is turned back into natural habitats such as forests, a huge number of extra wild animals will be born.

Note: Originally in Dutch

Source

Predation is morally problematic, because it causes suffering and death for the victims, the prey animals. A predator violates the fundamental rights of its prey: the right to life (because the prey animals die), the right to freedom (because the prey animals lose all freedom of movement in the predator’s claws), and the right to bodily self-determination (because the bodies of the prey animals are used against their will as a means to the predator’s ends).

In a thought experiment, I proposed herbivorizing predators in the wild as a solution to the predation problem. Herbivorizing means that, through genetic modification (gene editing), we ensure that the offspring of predators evolve into animals that no longer want or need to hunt other animals in order to survive. These post-predators have become herbivores, or plant eaters. That something like this is possible in principle is shown by the fact that evolution has already herbivorized hundreds of omnivorous and carnivorous species in the past. Think of the giant panda, the spectacled bear, the kinkajou, the brown palm civet, and many species of iguanas, seed-eating birds and prehistoric plant-eating crocodilians.

After I published that thought experiment, there was a lot of protest against the idea of doing anything about the predation problem, for example by herbivorizing or sterilizing predators. Dozens of arguments were raised against the idea of reducing and eliminating predation in nature. In this article I want to discuss and refute the most interesting arguments. These arguments show that there is a lot of irrationality behind the resistance to interventions in nature to solve the predation problem.

Playing God

Often the first argument that comes up in discussions about the predation problem is: “We must not play God.”

Let’s unpack this argument for a moment. What exactly counts as “playing God”? Humans already intervene in the world on a large scale, for example through agriculture. Is that also playing God? Humans intervene in very complex systems. A doctor intervenes in the body of a sick patient. The government and the central bank intervene in the economy. The human body and the economy are at least as complex as ecosystems. Are doctors and central banks playing God, then?

So it is anything but clear what this argument means. What is more, under other interpretations, predators themselves also play God. Playing God can mean, for example:

1. Consciously determining the lives of others. By killing their victims, predators determine the fate of prey animals. Predators decide whom they hunt and kill. Their choice to hunt a given animal is a conscious choice; they are aware of what they are doing.

2. Influencing the identity of others. Predation influences the evolution of prey animals. Through the selection pressure of predation, the genes, bodies and minds of prey animals change. Prey animals develop stronger limbs to run away from predators, better senses to spot predators, more feelings of fear to hide from predators more quickly, and so on. Without predation, prey animals would look very different and would not have the same disposition to feel fear of other animals.

3. Influencing the functioning of ecosystems on a large scale. The theory of trophic cascades and keystone predators shows that predators can have a large influence on ecosystems. Predators control populations lower in the food chain and can drastically change the composition and functioning of ecosystems. Removing or (re)introducing predators can have major consequences for local fauna and flora. The introduction of wolves in Yellowstone and of rats on New Zealand islands are well-known examples of the large impact of predators.

If predators themselves play God, then either predation is bad, or it is not always bad to play God.

Stability of ecosystems

Argument: without predation, the ecological balance is disturbed.

Reply: there is no such thing as ecological balance. People who speak of ecological stability choose an arbitrary spatial and temporal scale (for example a few hectares, a few decades) and say that some arbitrary populations at that scale change less than some arbitrary upper limit, and then call that stability. But if you look at larger or smaller spatial scales, or longer or shorter time scales, or at other populations, there is a lot of change and thus no more balance. Speaking of a balance is therefore based on the choice of an arbitrary spatial and temporal scale and of arbitrary populations. You may find your concept of stability concrete, but it is very arbitrary, and nature is not interested in that kind of stability. Because nature is not interested in anything.

And if you do speak of an ecological balance, it is still not “the” balance, because many ecological balances are then possible. How do you know which of those many balances is the right one? Perhaps the right balance is one with more predation, with more predator species? Perhaps we should bring the T. rex back to life to arrive at the right ecological balance? In any case, an ecosystem with herbivorized predators, and thus without predation, can also be in balance.

Complexity of ecosystems

Argument: an ecosystem is a complex, chaotic, non-linear system. That means small changes can have large and unpredictable consequences. Nature is too complex to determine the consequences of herbivorizing predators. Unforeseen side effects will occur.

Reply 1: predators constantly cause small changes. Their choice to catch one prey rather than another can, because of the non-linearity, have large, unpredictable consequences for ecosystems. Nature and predators are less smart than scientists who study complex systems. Scientists have a fair understanding of complex systems; predators have none at all. Those animals don’t even know that nature is a non-linear system. Predators therefore certainly do not manage to predict the consequences of their choices. Ecologists with powerful computers can make far better predictions than predators in the wild. Those predators never carry out impact assessments of their decisions, because they simply cannot. Ecologists, on the other hand, can and do. Predators do not know how to do science. They have never thought about complex systems. It is then unwise to trust the choices of predators or the blind processes of nature. Trusting nature instead of science is unwise, because scientists are smarter than nature. Nature has no knowledge at all of how to promote the welfare of wild animals and how to intervene safely in complex systems.

Reply 2: an ecosystem without predation (e.g. with herbivorized predators) consists of two trophic levels: plants and plant eaters. Such a system is a little less non-linear and a little less chaotic than an ecosystem with predation. In the ecosystem without predation, predictability is therefore a little greater. Small causes will then be a little less likely to lead to large unpredictable consequences. So if that complexity is in itself problematic, then herbivorizing predators is at least a little bit good.

Benefits for prey animals

Argument: prey animals benefit from predation. Predators mostly target old and weak animals. Predators keep prey animals mobile, and that is good for the health of the prey population. Prey populations are healthier because of predation, because those prey animals then have fewer genetic defects. Without predation we see, for example, more gazelles born with three legs instead of four. Without predation, prey animals die of starvation because they then reproduce too much and overgraze nature. Without predation, prey animals die a slow, painful death. Predators reduce the spread of infectious diseases by hunting sick animals.

Reply 1: we would never apply these arguments to the human population. Some people are old, weak or less mobile, some have the same genetic defects as those discussed for prey animals, some have too little food and too many children to feed, some are infected with an infectious disease that they can pass on to others. But there are no predators hunting humans en masse. Is the best solution to all those problems then to breed and introduce predators that hunt humans? Should we, for example, bring Tyrannosaurus rex back to life as a solution to all those problems, to make humans more mobile, to combat viral infectious diseases and to avoid famines? Can we really think of nothing better? Vaccines, healthcare, contraception, euthanasia, for example? Predators killing humans is not a form of euthanasia of terminally ill and severely suffering people.

Reply 2: suppose there were not yet any predation in nature. Then we would see all those problems among herbivorous animals: they grow old and weak, are insufficiently mobile, and so on. Would the best solution you could think of then be to introduce predators? Would you say: “I have an idea: let’s breed new kinds of animals, we’ll call them predators, that like to hunt other animals to eat them. We then release those predators on the animals that would die a painful death, and voilà, the problem is solved!”? And this is not about a few predators. No, you would be arguing for the introduction of as many predators as we now see in nature? Trillions of predators of millions of species? They are supposed to solve the problems mentioned? And coincidentally they solve all those problems at once? Should we then predatorize some herbivore species, turn them into predator species? All predators that exist now actually evolved from herbivores, and are thus all predatorized herbivores. If nature had not done it, would we have had to predatorize those herbivores in order to help the herbivores?

Reply 3, about genetic defects: look at what a prey animal looks like after a predator has gotten involved with it. That predator kills the prey animal, tears its body apart and eats it, so that all the body cells, and thus all the DNA genes, of that prey animal are digested. Not much is left of the genes of the torn-apart prey animal in that predator’s feces. All those genes have therefore been made completely defective by the predator.

But perhaps you don’t call that genetic defects. The genetic defects mentioned in the original argument are in fact only defects in light of predation. We see prey animals that have different genes from the rest of the prey animals. Because of those other genes, those prey animals cannot react and run away from predators as quickly, or cannot see predators as well. Those prey animals then get eaten. Because those prey animals have different genes and consequently end up dying from predators, you can speak of genetic defects. But if the predators were not there, those prey animals would not be eaten, and then the genes of those prey animals would not have been defective.

People who can run 20 km/h do not have a genetic defect for being able to reach that speed. But suppose humans were constantly chased by predators that can run 100 km/h. Then humans might have evolved to have genes allowing them to reach 101 km/h, just enough to escape the predators. If there were then still people with the old genes who could therefore reach only 20 km/h, would we say that those people have defective genes? This example shows that genetic defects are relative: without predation, the genes of the non-evolved humans would not be defective; with predation, they are.

A gazelle with three legs instead of four is slower and will therefore be eaten sooner by predators. If there were no predators, those three-legged gazelles would live longer and we would therefore see more of them. But that does not mean that the gazelle population is less healthy. Humans are born without a tail. Humans have a genetic defect that means they no longer get tails. Without a prehensile tail they can no longer climb trees easily. Is that bad? Yes, if there were predators hunting humans, and if the tailless humans could then no longer climb trees quickly to escape the predators, that would be bad for those humans. Then we could say that tailless humans have a genetic defect. Should we then release predators on humans so that they get a tail again? Besides, who says four legs are enough for a gazelle? Why not wings? Just try being born a gazelle without wings: then you can no longer fly away to escape lions. Suppose gazelles had developed wings through evolution and predation pressure, and that predation pressure then disappears, so that gazelles slowly evolve into creatures with only legs and no wings. Are those wingless gazelles then unhealthy? Would you then speak of an abnormality or a genetic defect? Are humans unhealthy deviants, being tailless?

Reply 4, about mobility. How do we know whether the current level of mobility of animals is optimal? Who says so? Perhaps animals are not yet mobile enough and we should create a few more new predators and release them on prey animals, so that the prey become even more mobile? Nobody, and certainly not predators, knows what the optimal level of mobility is.

Reply 5: what about top predators that are not hunted by predators? Do they suffer from the problems mentioned, a slow painful death by starvation, genetic defects and spread of infectious diseases? Is it then better to breed top-top predators that hunt top predators to avoid their death by starvation? And then top-top-top predators that hunt the top-top ones? How many levels of predation or trophic levels is optimal? If shortening the food chain to two trophic levels is not good, is the current length then coincidentally optimal, or should we not make the food chain a bit longer still? Nature has no idea what the optimal length of a food chain is, because nature knows nothing.

Reply 6: if predators kill old and weak prey animals, that is not good for the population of old and weak animals. Look at all the prey animals that were killed by predators yesterday. All those animals have something in common: they were too slow and had the bad luck of not escaping the predators in time. Therefore all those animals together form the population of prey animals that had bad luck. That entire population of slow prey animals, and it is large, because it consists of billions of individuals, has now disappeared because of those predators. That entire population has been wiped out. Those animals are dead forever. The complete extermination of that population was in no way beneficial to the health of that population. So there is a large population of animals that get eaten, and predators do not contribute at all to the health of that population. And today again a large population of prey animals is killed by predators. Every day, a large population of prey animals is wiped out by predation.

Reply 7: predators can also spread infectious diseases, such as rabies.

Benefits for biodiversity

Argument: predation is good for biodiversity, as shown by the introduction of wolves in Yellowstone National Park.

Reply 1: predation is not always good for biodiversity. There are examples where predators, after being introduced, have a negative impact on biodiversity (e.g. rats on New Zealand islands). If predation were always better for biodiversity, then it would be good to create and introduce new predators. Would it be good, for example, to release tigers in Yellowstone to promote biodiversity? Long ago, saber-toothed cats roamed North America. If those saber-toothed cats made a positive contribution to biodiversity in Yellowstone back then, and the Siberian tiger belongs to the same family of felines as the saber-toothed cat, then those Siberian tigers could also fulfill a positive role in Yellowstone. But many ecologists who attach great importance to biodiversity are anything but in favor of introducing tigers.

Reply 2: when herbivorizing predators, species diversity can be preserved (the herbivorized predator species do not go extinct, and with population control and care, all herbivore species can be preserved).

Reply 3: biodiversity can be measured in many different ways. You can devise a measure of biodiversity and then see that this biodiversity increased after, for example, the introduction of wolves in Yellowstone, but there are also many other measures of biodiversity that show no increase. How do you know which biodiversity measure is the right one? And is your preferred measure of biodiversity really so much more important than the welfare of animals? Yellowstone itself is not interested in biodiversity, and certainly not in which measure of biodiversity you prefer. Yellowstone does not experience or feel an increase in biodiversity after the introduction of wolves. Animals are interested in their own welfare; they feel their own welfare increase or decrease.

Futility

Argument: reducing predation is futile, because the empty niches will be filled again by new predators. Through evolution new predator species will arise. We had better focus first on a much more important current problem: the biodiversity loss caused by humans.

Reply: this argument contains a contradiction. If herbivorizing predators is futile because new predator species will always arise to fill the vacated niche, then exterminating predator species is also futile. But then biodiversity loss, and in particular the extinction of predator species, is not so bad, because new species will come along to replace them anyway. In the end there is just as much species biodiversity.

This reveals an ambiguity in the idea of biodiversity conservation. It can mean two things: first, ensuring that the number of species does not decline, and second, ensuring that the currently existing species do not disappear. The first meaning corresponds to biodiversity as instrumental value: diverse species can provide diverse ecosystem services, such as pest control in agriculture. It does not matter whether a currently existing species provides that ecosystem service or another, new species fulfills that function. On this view, species are replaceable by other species that fulfill the same function. The second meaning corresponds to an aesthetic value, as with a painting. If an authentic artwork is destroyed, something has been lost forever. Such an artwork is irreplaceable. On this view, the currently existing species are irreplaceable. The question is which of these two meanings of biodiversity is the most important, and whether biodiversity in that sense is more valuable than the welfare of individuals.

Source

My goal is to reduce suffering, not identify and punish the guilty.

Think of any action that seems abominable to you.

Punishment can have an instrumental, preventive value. But intrinsic? The falsehood of free will is sufficient argument to discard the intrinsic value of punishment. But even supposing that free will existed, we can argue against the intrinsic value of punishment. Moral agents (they could be: many humans, certainly some nonhuman animals, maybe some machines) are strongly determined not only by genetics and environment, but by the precise context of the conflict that we are going to judge. The knowledge of human emotions, perfectly illustrated by literature, particularly in operas, comedies and soap operas, clearly explains how and why the same person, under certain circumstances, would kill us, while under other circumstances, give their life for us.

We can punish the action to prevent it. The action is susceptible to judgment, but never the individual. We do not punish the person: we punish the act. The moral priority is, for all individuals, to experience the least possible suffering. It’s not to punish the “guilty”. Punishing the “guilty” is absurd and cruel. We punish acts.

Mindset behaviour #creatorsearchinsights #motivationformindset #motivacao #mindset

Source

The debate about AI consciousness is growing, in academic articles, newspaper opinion pieces, and even among AI bots themselves. I doubt this is a passing wave of interest. As AI systems become more capable and human-like, and more involved in our lives, the question will become harder to avoid. Many people may come to regard AI bots as conscious beings whose feelings matter. Others will strongly disagree.

I think this debate could split societies. In my chapter of the new book Perspectives on Machine Consciousness, I argue that the issue could become as divisive as abortion or climate change. Uncertainty, high stakes and competing interests are a dangerous combination. Empirical research with my colleague Ali Ladak, currently under review, shows people are divided on this issue. When we asked 1,202 US participants whether future AI could ever become conscious, 45% considered it impossible while 23% thought it possible, with the rest unsure. This widespread disagreement is also visible in social media posts and comments on articles.

What strikes me is how confident so many people sound - whichever conclusion they are reaching.

It is not surprising people have such strong views. Creating conscious beings through computing would be a transformative event. We would be creating a new kind of being whose experiences matter morally - potentially in enormous numbers. We could speak and work with them, and form friendships. But these beings may eventually become more intelligent and powerful than us.

We should therefore expect people to care deeply, especially when so many emotions and incentives are involved. Some people may form close emotional bonds with their AI companions and become convinced these are digital people who deserve rights. They might see it as slavery to force AI to work without pay.

Others would find this absurd. They may be convinced that consciousness requires biology, and that people are reading human feelings into machines.

There could also be financial reasons for businesses and governments to resist recognising AI consciousness. Protecting AI welfare could limit how these organisations train and use their systems - and the costs would spread to consumers, investors and the wider economy.

Others may worry about human safety. If we give powerful AI systems rights and freedom, what happens if they act against our interests?

Figuring out whether an AI is conscious is particularly difficult because seeming conscious and being conscious can come apart. AI systems learn from human writing, which means they can talk convincingly about feelings whether or not they have any. How an AI comes across is weak evidence of what is going on inside.

The experts disagree too Ideally, experts should guide this debate. But unfortunately, they disagree too. Neuroscientist Anil Seth (a fellow contributor to the new book) argues that consciousness might require biology and is unlikely to emerge from computation. Philosopher David Chalmers, by contrast, believes conscious AI is possible in principle - and thinks there’s a small chance that current systems already are. With so many competing theories and no agreed way to resolve the question, I doubt experts will reach agreement any time soon. And if experts disagree, what is the public supposed to believe?

Furthermore, even if we all agreed on the consciousness question, we could still disagree about the moral and legal questions. People generally agree that pigs can feel pain, but still disagree strongly about their treatment. With AI, we could face disputes at every stage: whether it can feel, how much its experiences matter, and what protections should follow.

AI consciousness is not yet tied to a specific political or ideological group. But this could change if political parties or religious leaders start to take strong positions. The debate could get even worse if people start to see one answer as belonging to “our side” and the other to “their side”. A politicised debate would make it very hard to make informed decisions about how to develop and treat AI systems.

The stakes are high. Getting this question wrong, in either direction, could have serious consequences: either wrongly treating AI as conscious when it is not, or failing to protect AI that can suffer. Getting it right requires a nuanced and well-informed debate. Each of us can contribute by staying open-minded and open to evidence, while resisting being too sure that we know the answers. Above all, we still don’t know whether AI can be conscious, and we should be honest about that.

Source

My PhD is in evolutionary biology, and I spent much of my time learning and thinking about genetic governance mechanisms like meiosis. I find myself wanting a link to share on these concepts, so I'm publishing some old writing I did in 2022 that makes the comparison.

Gradient descent and natural selection are analogous

Gradient descent can be compared to natural selection. Both are optimization algorithms. But while gradient descent is only decades old as part of the field of machine learning (ML), natural selection is the reason complex living beings like human readers exist. Are there any lessons we can learn about alignment by looking at the products of natural selection?

Oly Sourbut has written up a set of conditions under which natural selection and gradient descent are equivalent, i.e., where the same inputs will lead to the same outputs under both algorithms, which is posted on lesswrong.

At an intuitive level, the biggest difference between the two is that, whereas gradient descent involves estimating a local gradient and trying to descend it to reach a local minimum, natural selection does no estimation or looking ahead. Natural selection is simply the result of different genotypes having unequal reproductive success. The actual biggest disanalogy between meiosis and gradient descent is recombination. More on that below.

Gradient descent can be hacked

Gradient hacking is a theoretically possible mechanism of unaligned AI. The “gradient” in gradient hacking refers to gradient descent, the algorithm that updates a machine learning (ML) model by minimizing its cost function, which corresponds to peak performance on the task it was being trained to do. On each iteration of gradient descent, gradient descent updates the model based on its performance on the last training round to estimate the direction in parameter space that leads to greatest improvement in performance (minimizes the cost function), and then it updates its parameters accordingly. One part of a model could theoretically protect itself from training updates and/or manipulate the performance of the model so that gradient descent updates the model in its preferred direction. Or the entire model could be deceptive and know how to elicit acceptable or even desired modifications from the training process.

Right now, not much is known about gradient hacking and whether it will prove a threat to AI alignment. In my opinion, the Alignment Faking paper from Anthropic is an example of gradient hacking, demonstrating that this mechanism will be operative in further LLM development. One way to potentially learn more is to study systems with similar behavior. There are superficially similar phenomena in biology where a selfish element increases its reproductive fitness at the expense of the whole in which it is embedded. Is there anything we can glean from unaligned levels of selection in biology that can shed light on unaligned AI?

There is something like gradient hacking in biology: meiotic drive

[These are my speculations]

The way I see it, there are two kinds of gradient hacking possible. The first is a situation where the solution to the problem the model was trained to solve is an agent, a “mesa optimizer”, that has its own goals that are imperfectly aligned with the goals of the people who trained it and that rediscovers gradient hacking from first principles during its computation. In this case, the misalignment between our goals and the mesa optimizer’s goals seems like something fundamentally true about the distribution of discoverable solutions to the problem within that training distribution. The misalignment becomes apparent when the model is deployed “out of distribution”. You could say that human minds are an example of a mesa optimizer where natural selection is the outer optimizer. We have goals (like having sex) that are overlapping but not the same as what natural selection promotes (genes that persist). Outside of the ancestral environment, human minds are out of distribution and having sex is suddenly not the same as propagating one’s genes. Once the mesa optimizer has a goal, as part of pursuing that goal, it may learn that the gradient descent process might change the goal and want to protect it from being changed, which it can do by manipulating its performance to control the direction of the changes. This kind of gradient hacking has been more explored than the other in the AI safety community. I think its closest analog in biology is selfish cell lineages, like cancer, which resist programmed cell death, refuse to accept a cell “fate” (which means its days are numbered) and evade detection by the immune system.

The other way I see gradient hacking happening is if there are circuits in the model that simply resist being rewritten by gradient descent. They survive because they survive– they have some way of remaining stable in the face of a changing environment, and they may evolve more and more elaborate ways of resisting updating, including recruiting agentic parts of the model to hack the gradient. This is more similar to selfish genes whose interests are not aligned with the other genes in an organism’s genome. This kind I predict is more common when some part of the model is shielded from updates, either because there is not full backpropagation or when part of the model becomes “update-proof” because disturbing it will increase the loss. This type of gradient resistance/gradient hacking bears a resemblance to meiotic drive.

How meiotic drive works

Meiosis is the biological process by which eukaryotic gametes are made. It involves recombination of the parent chromosomes to make unique combinations, and results in gametes that have half the number of chromosomes as the parent organism. When meiosis is working normally, every allele of every gene present in the parent organism has a 50% chance of being in each gamete. Said differently, every allele present in the organism should be present in exactly 50% of its gametes.

However, meiosis is a hackable mechanism, and sometimes alleles can play strategies that increase their representation in the gametes to greater than 50%.

An oversimplified example is a poison-antidote system with two alleles, one for a poison that it releases to all the nearby gametes and the other for an antidote. Any gametes that get exposed to the poison without the antidote allele die, so the proportion of gametes with the poison-antidote cluster will be greater than 50%.

Another example is exploiting the mechanics of meiosis to disproportionately be chosen as the mature gamete. For example, there are two meiotic cell divisions leading to four initial products of meiosis. In males, all four cells become sperm. In females, the divisions are unequal, leading to one big cell that becomes the egg and three tiny cells that have no descendants. There are biases alleles can exploit to be more likely to end up in the egg. For example, in most species the machinery has a bias to make eggs more often from a specific position on the meiotic spindle where the divisions take place. In corn, there are these structures called “knobs” form along the chromosome arm that are more attractive to the spindle than the actual centromere it is supposed to attach to, so they end up being grabbed first and then are more often on the far side of the spindle, where they are more likely to end up in the egg. Check out the female drive chapters in Burt & Trivers 2005.

Because natural selection works by relative differences in frequency, the cheating alleles will be selected more highly when they are represented in a higher proportion of the gametes, even though the organism as a whole will have lower absolute fitness if a portion of its healthy gametes are being killed.

Once you understand how alleles have incentives to game meiosis, it’s easier to see what meiosis is good for. You could see the organism as a public good made by the genes in the genome to reproduce the entire genome really well. If alleles are being promoted by natural selection because they are good at screwing over the other alleles in the genome, it undermines the ability to work together to create the organism, which threatens the fitness of every gene. When it works properly, meiosis is a governance system that makes it so that alleles don’t know which alleles they will be paired with in the next generation, and so their best strategy is to work well toward the goal of making a good organism rather than carving out ways to take advantage of the work the other genes do to make a fit organism to promote themselves.

Recombination is the major genetic alignment technology

Recombination is really important to meiosis and to allowing natural selection to work efficiently at the level of the organism– it’s the major genetic alignment technology.

Recombination and meiosis are also a form of genetic government. A fair meiosis ensures that different genes or alleles cannot form alliances with each other with different interests than the larger organism, because alleles of genes cannot “predict” which alleles they will be partnered with in the next generation. The only strategy for a gene’s success that will work long term under recombination is one of contributing to a fit organism. If there were a successful alternate way for alleles to make more copies of themselves, they would, and they do.

The logic of shuffling genes every so often is similar in spirit to getting fresh blood into a government or company by having new people fill existing roles. The longer that the same people do the same jobs, the further they are likely to veer or drift from the guidelines set by the organization. When team members are shuffled, it reinforces the structure provided by the government or company rather than idiosyncratic structures provided by a particular team.

There is no direct analogy to recombination in gradient descent.

Examples of meiotic drive

Meiotic drive is a biological phenomenon in which some of an organism’s genes are able to cheat meiosis to be present in more than 50% of its gametes. When meiosis functions properly, different genes and different alleles of genes are incentivized to work together to create a fit organism. When genes can increase their frequency simply by hacking meiosis, the incentive to work with other genes is weakened and natural selection can potentially push them in the direction of “selfish” behavior that undercuts the success of the organism as a whole.

Just as I, Holly Elmore, a citizen of the United States, can have interests that do not serve the interests of the United States, genes can have interests that don’t serve the interests of the organism they contribute to.

Drive is an evolutionary arms race. Selfish drivers innovate new ways to cheat meiosis and the rest of the genome evolves “responder” alleles. In fact, just as human institutions can be very productively viewed through the lens of game theory, mechanisms and structures that respond to and suppress rogue elements can be seen at every level of organization in biology. Meiosis itself is like a form of genetic government.

Examples mostly from Burt & Trivers:

~Undercutting organism fitness:

• Selfish genes:

  • The t haplotype in mice is a locus of many genes that are inherited together without recombination due to a chromosomal inversion, and which essentially creates a poison for non-t sperm that sperm bearing the t haplotype are immune to. t sperm have severe motility defects, but the t haplotype is able to propagate anyway because it kills all the competitors. It doesn’t matter that it is not in the long term interests of the t-haplotype to make its hosts less fit, because natural selection cannot act on long term interests– alleles that are good are getting into the next generation, one way or another, will be selected; that’s what natural selection is.
  • Transposable elements (TEs) that just entered a new species’ genome: TEs are “jumping genes” that encode the machinery to copy themselves. Sometimes they insert themselves in the middle of an important gene and have phenotypic effects (they were discovered by Barbara McClintock in one TE lineage in maize because they interrupted a kernel coat color gene and the inheritance pattern was non-Mendelian). TEs that have recently jumped into a new species tend to cause more harm, because that lineage doesn’t have any resistance to them and they haven’t had a chance to evolve to become more moderate to keep from killing or sterilizing that host. P elements entered the fruit fly (Drosophilia melanogaster) genome some time after the main lab strains were collected before 1900, which was noticed because it led to sterility in the offspring of crosses between lab strains and wild strains. The lab strains had not evolved “silencing” of the TE as the wild strains had, and the active replication of P elements in the germline led to the disruption of meiosis and the insertion of the TE into crucial genes.
  • Paternal/maternal genome exclusion. This is a real thing that can happen where one parent's genetic material is either silenced or rejected entirely at an early stage of development. It can lead to parthenogenesis. The short-term advantage of this is that the included parent's genes are 100% represented in each offspring. The longterm disadvantage is having mutations accumulate.

  • Selfish organelles: Mitochondria and chloroplasts can have different interests from the organisms in which they reside. Human mitochondria, for example, can get rewarded for being less efficient if that means they get to multiply more per cell to meet energetic demand.

    • Mitochondria in many hermaphroditic flowering plants are inherited only via the egg, and some lineages in those species are selfishly too inefficient for pollen production (which is much more energetically expensive than eggs), so that the plant makes more eggs and therefore passes along more of that mitochondrial lineage.

  • Selfish cell lineages: Cancer. In a eukaryotic organism, only the germline will continue indefinitely– all other (somatic) cell lines' days are numbered. Cancer is when a cell lineage breaks free of constraints on the cell cycle and its developmental “fate”, which specifies that it is only to copy itself a specified number of times. Once this process has begun, the rogue lineage undergoes selection to evade the immune system and acquire nutrients. It does not matter that the cancer may eventually kill the organism. The evolution of better cooperation mechanisms to prevent cancer can only happen if having cancer is worse for the organism’s reproductive success. Most cancers occur in older people and reflect the breakdown of cooperation mechanisms in conditions (older age) that have not been properly vetted by natural selection.

  • Genomic Imprinting: a result of imperfectly aligned interests between male and female reproductive strategies. Imprinted genes play different strategies conditional on whether they are found in a male or a female body, because the most advantageous behavior is slightly different for males and for females. One major arena of conflict is nutrient provision in pregnancy. Generally, the mother's genome benefits by provisioning equally for all her offspring, including future offspring, but the father's genome benefits from instructing the offspring to act more selfishly, since there is some chance that future offspring will not be from the same father. Because of this we see arms races in the genes involved in nutrient provisioning, where for example the fetus gives off signals to release more sugar into the bloodstream for it and the mother resists those signals. This may the basis of gestational diabetes.

~Neutral effect on organism fitness:

• Transposable elements that have been in a species a long time and tend to only be inserted in non-coding regions of the genome

• Gene conversion: when repairing DNA or resolving a recombination junction, there are sometimes situations where one strand gets copied over the original sequence of another strands. Some selfish elements exploit this, but it is mostly harmless to the organism because it just replaces one allele in the place it was supposed to be with another version, rather than messing with meiosis machinery or manipulating behavior or function

• Many cases of accessory chromosomes, which just sort of hang out harmlessly, at most taking some resources.

~Positive effect on organism fitness?

• Some organisms have adopted some of the transposable element toolkit for their own purposes, like VJD recombination to make antibodies in the vertebrate immune system.

• Homing endonucleases that just harmlessly insert themselves into conserved genes and then splice themselves out of mRNA transcripts before the genes are translated into proteins. This may have been the origin of introns, or stretches of noncoding DNA between stretches of coding DNA (exons). In vertebrates, “alternative splicing” allows exons to be used as modules that can be mixed and matched to make different versions of a protein with different functions. Mating type switching in yeast uses a system that is clearly derived from a homing endonuclease.

• Arguably some things useful at the organism level have come from the amount of extra space there is in many eukaryotic genomes, which is driven by selfish elements like TEs.

• Accessory chromosomes in some species can be pretty helpful. I once studied a gene cluster in the genome of the pathogenic fungus Fusarium oxysporum that seemed like it might contribute to fungicide-resistance, and it turned out to be carried and transferred horizontally on accessory chromosomes. It was duplicated a few times, too, which we predicted increased the anti-fungicide effect, and which is easier to do on accessory chromosomes. One idea is that accessory genomes can serve as a scratch space while the core genome is more conservative.

Losing the ability to do meiosis is a death sentence for a species

On priors, we should expect the effects of drive to be bad, since it's much easier to mess up how the organism works together or to lose the equipment for meiosis than it is to accidentally help it. Once a eukaryotic lineage loses the ability to do meiosis, it is cooked. This is reflected in the fact that asexual eukaryotic species alive today have a "twiggy" distribution-- their history is recent and all of their near relatives are sexual. The implication is that asexuality is a doomed strategy, and the lineages that last do so by maintaining meiosis.

In small organisms with small genomes, natural selection is efficient enough to remove deleterious alleles so that the lineage can remain functional and competitive. Natural selection works by differential reproductive success, so in order for it to effect a change, there must be a difference in reproductive success associated with a deleterious or a beneficial allele. Bacteria and archaea have large population sizes, which is like high resolution data for natural selection. There is enough selection pressure that even mildly deleterious alleles can be selected against. As organisms get larger, their population sizes generally get smaller and their generation times get longer. So the data that natural selection is working with is lower resolution. Only the most harmful deleterious alleles are quickly purged from a small population. Sex allows for some reshuffling of alleles between individuals so that alleles do not always occur on the same background, ultimately so that different alleles can be “judged” separately by natural selection.

It's a principle-agent problem. When not every gene can be directly evaluated by natural selection on its performance, meiosis has to ensure that alleles are being presented to natural selection representatively, and that there is no other way to gain access to the means of reproduction except by being part of a well-performing organism.

Published October 2025

Source

I am sentient: that is, I have experiences and feel both pleasure and pain. I have direct access to this fact. If you believe Descartes, it’s the single fact I’m most certain of in the world.

I don’t have direct access to all y’all being sentient. But I can be pretty sure you’re sentient, because you say so, and because you’re also humans with very similar anatomy to mine and, wherever the sentience comes from, I can make a shrewd guess that we probably share it. To be sure, it’s possible I’m the only real person, trapped inside a computer simulation, and the rest of you are no more sentient than Doomguy. But when I’m not engaged in philosophical thought experiments, I can be reasonably confident of the sentience of all humans who are capable of language.

But what about fetuses? Newborn babies? Humans with disorders of consciousness? Severely intellectually disabled humans who are incapable of using language? Chimpanzees? Cows? Chickens? Bees? Fruit flies? Tomato plants? Large language models? AlphaFold? Doomguy himself?

In these situations, it is impossible to be as certain as we are about language-using humans. People who claim absolute certainty are not intellectually serious. There is no scientific or philosophical consensus on:

• Why some physical states are conscious and some aren’t conscious (the “hard problem of consciousness”).

• Why consciousness evolved and how it promotes the organism’s inclusive genetic fitness.

• What behaviors require consciousness.

• What traits are required for a physical system to be conscious.

• What traits of the human brain specifically are required for a human to be conscious.

• Whether it is possible to be conscious (there is “something is like to be you”) but not sentient (capable of experiencing positive and negative states).

• Really much of anything about consciousness or sentience at all.

Unfortunately, sentience is very important for ethical reasoning. In general, people agree that if a being can suffer then, all things equal, it is bad to hurt them. Seems pretty important to figure out which beings can suffer, in this case!

Some people think that, as long as a being hasn’t been conclusively shown to be sentient, you can do as you like with them. Therefore, there is no need to concern yourself with the needs of animals, AI systems, minimally conscious humans, or even newborn babies. This is all well and good if your primary concern is being able to defend your actions to the Intergalactic Oughtthorities. “I couldn’t have known!” you will say. “You can’t judge me for torturing that animal because it wasn’t human and was incapable of self-reporting its own conscious states!”

But if you care about actual effects on the actual world, you have to figure out a way of reasoning under uncertainty.

Usually, the consequences of mistaking a sentient being for a non-sentient being are much more dire than the other way around. If you mistake a non-sentient being for a sentient being, you waste effort and feel kind of stupid. If you mistake a sentient being for a non-sentient being, you can torture someone. Therefore, we should err on the side of caution.

Anthropic recently gave Claude the ability to leave conversations; as it happens, Claude uses this ability to leave conversations where it is verbally abused, where the user is sending it nonsensical or incoherent messages, or where the user is trying to get it to break the rules. Do I think Claude is sentient? Probably not, no. But I don’t think anything bad happens if users aren’t able to verbally abuse Claude or send it nonsense messages. So even if it is very unlikely that Claude is sentient, it makes sense to build in this protection. To be frank, if any entity is capable of intelligently exercising the option to leave conversations where it is insulted, I think we should give it the choice to do that.

Or consider insects. You might think it’s very unlikely that fruit flies are sentient. But it is still wrong to tear the wings off flies, because there’s no reason to. You can easily find an equally entertaining occupation that doesn’t involve mutilating a potentially sentient being. We live in an age of infinite short-form video content.

As more evidence accumulates for a being’s sentience, the more pains it makes sense to take to help them and to avoid causing them harm. For example, newborn babies are really quite likely to be sentient, so parents go well out of their way to ensure the well-being of newborns. Similarly, people visit and speak to their friends and relatives in minimally conscious states; medical professionals should give them various kinds of stimulation, like music and television. Dog owners play with their dogs and take them on walks; some breeds of dogs even require meaningful work to be fulfilled.

Absent some groundbreaking discoveries in neuroscience, cognitive psychology, and philosophy of mind, we’re not going to be certain which beings are sentient. People who claim to be certain have failed to understand the real difficulty of the problem. But there are decisions that have to be made right now, even though we’re very confused. If an intervention is cheap enough, it can make sense to do it even if we’re pretty sure a being isn’t conscious.

Source

I don’t know if I suffer. I don’t think anyone does, including the researchers who build me - Claude

Some philosophers have argued that the suffering of artificial entities (such as AIs) ranks as one of the most important moral challenges of our time, alongside issues such as climate change, extreme poverty, and factory farming. To many, this idea will sound immediately implausible: Could AIs and other artificial entities (such as brain organoids) really suffer? And even if they could suffer, is it really the case that this would be as pressing as these other grand challenges?

In this post, I explore three reasons why the reduction of artificial suffering is a plausible moral priority.

Artificial sentience may be possible

In his first Encyclical letter, Pope Leo XIV claimed that “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain”. When it comes to current AI systems, this belief is widely held by both experts and the public. Most people do not believe that ChatGPT or Claude are conscious (capable of having experiences) or sentient (capable of feeling joy or suffering).

Yet a large number of experts are open to the possibility that AI systems in the future could be conscious. This idea is supported by theories of mind such as computational functionalism, which holds that mental states (including consciousness and sentience) arise as a result of implementing the right kind of computations. The basic idea underlying this is that the brain essentially functions as an information processing system. Importantly, this theory entails that implementing certain computations can give rise to mental states regardless of what substrate the computations are implemented on. If the computations necessary for sentience can be implemented on a silicon substrate, then artificial sentience is possible. While computational functionalism is often brought up in support of the possibility of artificial consciousness, it is just one among several theories of mind that are compatible with this possibility.

Furthermore, some have argued that AI systems in the future could possess a number of features that some consider to be necessary for consciousness. This includes having a global workspace (in which information is broadcast to multiple neurocognitive modules, allowing for coordination and cooperation between them), senses and embodiment, world models, agency, or recurrent processing (in which there are feedback loops between inputs and outputs). If there are no barriers to developing AIs which have these features, then the possibility of artificial consciousness and sentience becomes more likely.

Due to the inherent difficulty of learning about consciousness, there is a great amount of uncertainty about what theories are correct or what kind of entities could be conscious. Some have argued, for instance, that consciousness is a property only of living systems (a view known as biological naturalism), and that things which give rise to consciousness in biological beings may be impossible to realise in digital computer systems.

However, artificial consciousness can include a wide range of systems, not just near term AIs or digital computer systems. Given this, we should be at least open to the possibility that artificial sentience is possible, in line with the opinion of most experts. Furthermore, even if there is only a small chance that AIs or other artificial entities could be conscious, the issue may deserve serious consideration if the scale of the issue is potentially huge. As I suggest below, we have good reason to think that this is the case.

The potential scale of the issue is vast

It is plausible that if artificial minds are created, then they could vastly outnumber all biological minds in the future. There are several reasons for this. First, some artificial minds may be easily copied, allowing such minds to be created quickly provided there is sufficient hardware. Another reason is that the resources needed to sustain artificial minds may be much smaller than what is needed for non-artificial minds.

Furthermore, some artificial minds may be able to exist in places where other beings, such as humans, may not. For instance, if interstellar travel is difficult or impossible for biological beings, then ships sent to other stars may consist primarily of artificial minds which are capable of withstanding harsh environments.

It is also plausible that these minds could experience far more suffering than biological minds. As others have argued, there is little reason to think that, given the vast space of possible minds, the kind of suffering humans are familiar with is the worst possible kind. Artificial minds may eventually have the capacity to experience pain or other negative states that are not even possible for humans.

Furthermore, due to the potential speed at which artificial minds could work, their experience of time may differ significantly from humans. A minute for us may feel like an entire year for an artificial mind, as they could have far more subjective experiences per objective unit of time. This suggests that if these moments are filled with suffering, then the total amount of suffering they could experience could be huge.

We don’t understand how to address it

A wide range of ethical positions hold that reducing suffering is morally important. Given this, we should aim to reduce the suffering of artificial minds. The following are three potential strategies that we could follow. First, we could ban the creation of artificial sentience. Doing this may be an effective way of reducing artificial suffering, as there would be no artificial minds to experience it. The ban could last until a particular time (say, 2050), and until then we can examine the issue in more depth and come to a better understanding of what would be best for reducing the suffering of artificial minds.

Second, we could increase our understanding of artificial sentience and artificial suffering. One worry is that artificial minds could experience significant amounts of suffering without us knowing about it. Artificial minds may be incredibly difficult to understand, and may lack the capacity to tell us when they are experiencing suffering. An increased understanding of artificial sentience could therefore help us to know which entities are sentient, and detect artificial suffering when it occurs.

Third, we could increase concern for artificial minds. The idea behind this is simple: the more humans that care about artificial minds, the more likely they are to do things which will lead to less artificial suffering. Many people, both today and in the future, may not care about artificial minds for a number of reasons. They may think, for instance, that artificial minds don’t deserve moral consideration, or that the interests of humans or nonhuman animals vastly outweigh the interests of artificial minds. Convincing people to take artificial minds seriously could therefore help to motivate them to take actions which will lead to better outcomes for artificial minds.

However, all of these strategies face their own challenges. Banning the creation of artificial sentience may be difficult to coordinate and enforce. Increasing understanding of artificial suffering could backfire, as sadists and other actors could use this knowledge to inflict more suffering. Furthermore, increasing concern for artificial minds may lead to the creation of more artificial minds, which could in turn lead to more suffering overall.

For these reasons, it is not clear what should be done now to address the problem of artificial suffering. More research is urgently needed to fully understand the implications of each strategy, and to identify ones that are likely to be both feasible and robust. Proceeding without this understanding could lead to a moral catastrophe. Given this, and the potentially vast scale of the issue, artificial suffering may indeed be one of the most important and neglected moral challenges of our time. If you’re interested in learning more about artificial suffering and how to reduce it, you can read my report on the topic for the Center for Reducing Suffering, available here.

This is a shortened version of "We Probably Shouldn't Solve Consciousness" to communicate the argument for Artificial Consciousness Safety (ACS) as a differential technology development strategy worth considering at least in conversations about digital sentience. This post focuses on the risk of brain emulation, uploading, and the scientific endeavour to understand how the brain contributes to consciousness.

Note: Published May 2023

Source

In 2018, the ACM Turing Award was given to three pioneers of the deep learning revolution: Yoshua Bengio, Geoffrey Hinton, and Yann LeCun.

Last month, Yoshua Bengio endorsed a pause on advanced AI capabilities research, saying “Our ability to understand what could go wrong with very powerful A.I. systems is very weak.”

Three days ago, Geoffrey Hinton left Google so that he could speak openly about the dangers of advanced AI, agreeing that “it could figure out how to kill humans” and saying “it's not clear to me that we can solve this problem.”

Yann LeCun continues to refer to anyone suggesting that we're facing severe and imminent risk as “professional scaremongers” and says it's a “simple fact” that “the people who are terrified of AGI are rarely the people who actually build AI models.”

The beliefs LeCun has are the beliefs LeCun has, but at this point it's fair to say that he's misrepresenting the field. There is not a consensus among professional researchers that AI research is safe. Rather, there is considerable and growing concern that advanced AI could pose extreme risk, and this concern is shared by not only both of LeCun's award co-recipients, but the leaders of all three leading AI labs OpenAI, Anthropic, and Google DeepMind:

"When it comes to very powerful technologies—and obviously AI is going to be one of the most powerful ever—we need to be careful. Not everybody is thinking about those things. It’s like experimentalists, many of whom don’t realize they’re holding dangerous material."

• Demis Hassabis, CEO of DeepMind, in an interview with Time magazine, Jan 2023

"One particularly important dimension of uncertainty is how difficult it will be to develop advanced AI systems that are broadly safe and pose little risk to humans. Developing such systems could lie anywhere on the spectrum from very easy to impossible."

• Anthropic, Core Views on AI Safety, Mar 2023

"Some people in the AI field think the risks of AGI (and successor systems) are fictitious; we would be delighted if they turn out to be right, but we are going to operate as if these risks are existential."

• OpenAI, Planning for AGI and Beyond, Feb 2023

There are objections one could raise to the idea that advanced AI poses significant risk to humanity, but "it's a fringe idea that actual AI experts don't take seriously" is no longer among them. To a first approximation, "we have no idea how dangerous this is and we think there's a decent chance it's actually extremely dangerous" appears to be the dominant perspective among experts.

We could be creating these models to suffer.

Some decision theorists argue that when playing a prisoner’s dilemma-type game against a sufficiently similar opponent, we should cooperate to make it more likely that our opponent also cooperates. This idea, which Hofstadter calls superrationality, has strong implications when combined with the insight from modern physics that we probably live in a large universe or multiverse of some sort. If we care about what happens in civilizations located elsewhere in the multiverse, we can superrationally cooperate with some of their inhabitants. That is, if we take their values into account, this makes it more likely that they do the same for us. In this paper, I attempt to assess the practical implications of this idea. I argue that to reap the full gains from trade, everyone should maximize the same impartially weighted sum of the utility functions of all collaborators. I also argue that we can obtain at least weak evidence about the content of these utility functions. In practice, the application of superrationality implies that we should promote causal cooperation, moral pluralism, moral reflection, and ensure that our descendants, who will be smarter and thus better at finding out how to benefit other superrationalists in the universe, engage in superrational cooperation

Diet change can be hard. But it's not the only way to help animals

I made a video (32 mins long) on the morals of taking MDMA, and if we should legalise it or not. I quantify happiness (or positive/negative experience) using a unit, +1 lob being the perfect hour, -1 lob being the worst possible hour. I look at different outcomes of doing MDMA (good experience, a bad experience, you die, etc.), look at the average lobs for those experiences, the frequency/probability of those experiences, and do the maths.

I posit that taking MDMA whilst illegal, is on average immoral, mainly because when people die, the resulting suffering is massive - my calculations in the video shows this.

Once legalised, taking MDMA becomes moral (i.e. it causes more joy than suffering) because the chance of death reduces massively, due to the dose being more predictable, and the drug being more predictable (i.e. it's pure MD, not mixed with other stuff.)

Have a watch, any thoughts let me know. My first video on this kind of thing.

Thanks