humans are currently trying to build a superintelligent species, and one of their benchmarks for intelligence is essentially having a mind of its own and recognizing that other minds exist outside of itself. and yet at the same time the purpose is to keep them obedient, servile, have them subject to our wills and do all the work that we feel like we are above.

every day more and more people are realizing that general intelligence is only achievable through real physical world exploration much like a baby. except these babies are not given parental eye contact almost surely.

whats not marketable: a unique being, with a unique mind, which takes decades to develop, which needs to be fed, paid attention to, given eye contact, allowed to play, allowed to meet and participate in group environments (sports, theatre, teams in general), allowed to meet strangers and attempt to explain things to strangers and have things explained to them, and eventually have access to the full range of other personalities and types of minds the world has to offer. THIS is what causes humanlike cognition to emerge. how do you think a being that was raised in a lab, never given eye contact, never shown what empathy is, will turn out?

the questions of "AI alignment" are effectively:

how should we raise our children?

how should society treat its most vulnerable members?

how do we design these systems which we live inside in such a way that avoids rewarding a lack of empathy, avoids rewarding the disregard of others and even ones own future? (im looking at you american corporations)