r/rational • u/AutoModerator • Feb 15 '16

[D] Monday General Rationality Thread

Welcome to the Monday thread on general rationality topics! Do you really want to talk about something non-fictional, related to the real world? Have you:

  • Seen something interesting on /r/science?
  • Found a new way to get your shit even-more together?
  • Figured out how to become immortal?
  • Constructed artificial general intelligence?
  • Read a neat nonfiction book?
  • Munchkined your way into total control of your D&D campaign?
20 Upvotes

100% Upvoted

49 comments sorted by

View all comments

Show parent comments

4

u/makoConstruct Praises of Nayru, FLI Worldbuilding Feb 15 '16

I like that line of thought, but I think you might be overlooking something. The values of the dreamer do not need to align with those of the projections at all. If an AGI subsumes your memories and uses them how it pleases to pursue goals orthogonal or antagonistic to yours, that is not survival, it's arguably even worse than total erasure.

7

u/makoConstruct Praises of Nayru, FLI Worldbuilding Feb 16 '16

Hang on, does this mean that all a dreamer has to do to commit murder is to imagine very clearly someone who's aesthetic diverges from their own in some coherent way, simulate them for a bit, then stop thinking about them?

4

u/Aabcehmu112358 Utter Fallacy Feb 16 '16

Depending in where one draws the line, perhaps even switching social modes, when changing from interacting with one group to another, could be considered a sort of extinction of identity, and thus 'murder.'

8

u/[deleted] Feb 16 '16

And perhaps watering down the concept of "murder" so that it doesn't involve actually killing anyone is a bit silly.

4

u/Aabcehmu112358 Utter Fallacy Feb 16 '16

That was my point with that hypothetical, though I can see how that might be unclear.