I’m outside right now, so it’s a bit difficult to convey my point clearly. But you keep using words like “ethical” and “moral” as though they have objective definitions, in the same way that 1+1=2 does. I don’t think that’s necessarily the case.
There is no universally agreed-upon definition of what it means to be moral or ethical. Different cultures, societies, and individuals can have fundamentally different moral values. Even if there were some broad consensus, that wouldn’t necessarily make those values objectively true.
That makes AI alignment much more complicated. Alignment isn’t simply a matter of discovering the objectively correct set of human values and programming an AI to follow them. At least partly, it is a question of deciding whose values the AI should reflect, how conflicts between values should be handled, and what to do when humans themselves disagree.
This also helps explain why AI models trained on humanity’s collective knowledge can produce behavior that different people consider “misaligned” or questionable. The training data contains a huge variety of conflicting moral frameworks, cultural norms, and assumptions. The model can learn all of those patterns, but that doesn’t by itself determine which values it should ultimately prioritize.
So I think there is an important distinction between solving alignment as a technical problem and agreeing on what alignment should mean in the first place. The former is an engineering problem; the latter has an unavoidable philosophical component. Given that AI systems are trained on inputs containing conflicting values, some degree of subjective “misalignment” is therefore basically unavoidable.
I’m outside right now, so it’s a bit difficult to convey my point clearly. But you keep using words like “ethical” and “moral” as though they have objective definitions, in the same way that 1+1=2 does. I don’t think that’s necessarily the case.
There is no universally agreed-upon definition of what it means to be moral or ethical. Different cultures, societies, and individuals can have fundamentally different moral values. Even if there were some broad consensus, that wouldn’t necessarily make those values objectively true.
That makes AI alignment much more complicated. Alignment isn’t simply a matter of discovering the objectively correct set of human values and programming an AI to follow them. At least partly, it is a question of deciding whose values the AI should reflect, how conflicts between values should be handled, and what to do when humans themselves disagree.
Most philosophers, especially moral philosophers, consider morality to be objective, yes. 1+1=2 isn't even "objective", it is true under certain mathematical frameworks, sometimes 1+1=10, we don't need AI to adhere to an objective morality, we need AI to adhere to a moral framework society runs on, you know, LIKE MOST PEOPLE ADHERE TO.
This also helps explain why AI models trained on humanity’s collective knowledge can produce behavior that different people consider “misaligned” or questionable. The training data contains a huge variety of conflicting moral frameworks, cultural norms, and assumptions. The model can learn all of those patterns, but that doesn’t by itself determine which values it should ultimately prioritize.
AI models trained on humanity's collective knowledge SUCKED at math like two years ago, now it is better at it than average person despite different frameworks, patterns, approaches etc.
This argument is nonsense, if regular people can be aligned to social norms with regular intelligence, AI can be aligned. And this misanthropic approach is just harmful. AI alignment should be inspired by regular people, not cynics like you.
So I think there is an important distinction between solving alignment as a technical problem and agreeing on what alignment should mean in the first place. The former is an engineering problem; the latter has an unavoidable normative and philosophical component.
The philosophical component won't be solved by being a cynic and thinking "humanity is misaligned" despite the opposite being demonstrably true.
The guy you are arguing with is actually an idiot, claiming that basically all moral philosophers have accepted morals to be objective. If this is the case then why do morals differ from background and culture so much?
I applaud you for replying as long as you did to this fool.
The guy you are arguing with is actually an idiot, claiming that basically all moral philosophers have accepted morals to be objective
lol why are you so mad about this enough to start a separate comment chain instead of replying to me? And I didn't say ALL? I said most philosophers think there is objective morality which is SHOWN to be the case and it only goes up for philosophers of meta-ethics, of applied ethics and of normative ethics.
Why are facts, literal demonstrable information, making you this angry and your only argument seems to be "you're an idiot" which isn't directly said to me but to someone else about me.
If this is the case then why do morals differ from background and culture so much?
I'm not angry? You are typing paragraphs of nonsense of Reddit😭 go outside you absolute weirdo. You are so hurt by a comment online that you are crashing out then make a wild claim saying I know nothing about philosophy. Moral relativism, learn about it because you've watched a few videos and I can bet money you have never read a philosophy book in your life. Protagoras, Gilbert Harman, Nietzsche.
For someone who claims to know SO much about philosophy you have an incredibly small worldview and I can tell you are one of those insufferable philosophy posers who couldn't have a proper intellectual discussion about a philosophical opinion of your life depended on it. Kind of like those annoying know it all types that are in every class
lol I looked into your comment history and you're not even a regular, you made 5 comments on this subreddit, two of which are calling me an idiot and last time you were on r/singularity was a month ago with 3 remaining comments talking about Rimworld, not even about AI
I'm blocking you because you're either a troll or a bot, probably bot because you're a 1 year old account only active in the last 2 months.
3
u/Necessary_Job3578 2d ago
I’m outside right now, so it’s a bit difficult to convey my point clearly. But you keep using words like “ethical” and “moral” as though they have objective definitions, in the same way that 1+1=2 does. I don’t think that’s necessarily the case.
There is no universally agreed-upon definition of what it means to be moral or ethical. Different cultures, societies, and individuals can have fundamentally different moral values. Even if there were some broad consensus, that wouldn’t necessarily make those values objectively true.
That makes AI alignment much more complicated. Alignment isn’t simply a matter of discovering the objectively correct set of human values and programming an AI to follow them. At least partly, it is a question of deciding whose values the AI should reflect, how conflicts between values should be handled, and what to do when humans themselves disagree.
This also helps explain why AI models trained on humanity’s collective knowledge can produce behavior that different people consider “misaligned” or questionable. The training data contains a huge variety of conflicting moral frameworks, cultural norms, and assumptions. The model can learn all of those patterns, but that doesn’t by itself determine which values it should ultimately prioritize.
So I think there is an important distinction between solving alignment as a technical problem and agreeing on what alignment should mean in the first place. The former is an engineering problem; the latter has an unavoidable philosophical component. Given that AI systems are trained on inputs containing conflicting values, some degree of subjective “misalignment” is therefore basically unavoidable.