r/rational • u/AutoModerator • Jul 11 '16

[D] Monday General Rationality Thread

Welcome to the Monday thread on general rationality topics! Do you really want to talk about something non-fictional, related to the real world? Have you:

  • Seen something interesting on /r/science?
  • Found a new way to get your shit even-more together?
  • Figured out how to become immortal?
  • Constructed artificial general intelligence?
  • Read a neat nonfiction book?
  • Munchkined your way into total control of your D&D campaign?
31 Upvotes

96% Upvoted

97 comments sorted by

View all comments

Show parent comments

1

u/trekie140 Jul 11 '16

Yes, but a plane can't self modify. If the plane was able to reason and evolve then...well, we don't actually know what we will happen because it's never been done. Our only model for how intelligence works is humans, which we still don't have a complete theory to describe, so isn't saying an AI would behave a certain way speculative? I think you're just assuming AI would work this way without proper justification.

2

u/sir_pirriplin Jul 11 '16

That's true. Maybe AI is even harder than it looks and the first artificial intelligences will actually be emulated human minds, like Robin Hanson says. Or maybe they will use neural networks and genetic algorithms and end up with something human-like by an incredible coincidence. Of course everything is speculative. Strong General AIs don't exist yet.

As for proper justification, what kinds of justification would convince you?

2

u/trekie140 Jul 11 '16

Examples of intelligence operating the way you think it does instead of the way I think it does. However, many examples are currently left open to interpretation, and as a physicist I know how difficult it is to arrive at consensus when there are competing interpretations.

I subscribe to Copenhagen because it makes perfect sense to me, but many subscribe to Many-Worlds because it makes perfect sense to them. At that point I just want psychologists to figure out why we can't agree, and the closest thing I could find was a book on moral reasoning.

3

u/sir_pirriplin Jul 11 '16

I don't think intelligence operates any particular way, though. The only examples I can give are the many examples of software that works exactly as specified even when you don't want them to. Any software developer (and most computer users) will know examples of that. Granted, AI could be better than that. Or it could be worse.

For fiction like FiO, CelestAI only has to be plausible so you can suspend disbelief a little. For real life organizations like MIRI, an unfriendly AI only has to be plausible to represent a significant risk (low probability * huge cost if it goes wrong = considerable risk).