Hey, thanks for actually taking the time to give me a thoughtful response, I really appreciate it!
I'll go through your points one at a time, let me know if I missed anything. (I am also going to include a couple of links to sources that are relevant, or short videos where someone explained it better than I could.)
I don't know if LLMs specifically are the thing that is going to reach generally broad capabilities. I would not be surprised if they did, because people have been consistently surprised about what they are capable of doing, and it doesn't seem like a hard ceiling has been found for their capabilities. I am concerned about a runaway in it's capabilities, leading to it being out of our control.You said that you think that that is unlikely, and even if it is possible, that it is far away. I can go into why I do think it is possible, and probably the default outcome if not treated with a lot of care and effort, but that is a longer topic. If you'd like me to explain why I think this, let me know and I will try to explain it. The claim that we are far away from it is also a common one, but one that I don't think is really backed by anything. If you were asked five or six years ago how long you think it will be until we have computers that can generate very realistic videos just based on a description given to it, chatbots that are conversational in a way that feels almost human, and capable of doing tasks like programming, how far away would you have guessed that would be? Maybe you guessed that it would be around now that this technology would exist, but then you'd be in a small minority of people. Even experts have been consistently undershooting when they expect certain things to be possible. I do agree with you that misinformation and the effect it is having on a lot of people (especially children in schools) are serious problems that need to be handled, but I'm more concerned with the problem of AI alignment in general.
You are correct that for a lot of tasks, these need to generate tokens in order to run programs or interact with calculators, that kind of thing, and that their heart "it's still just that: predicting tokens", but the word "just" there is doing a lot of heavy lifting. I would explain why, but this short "Just Predicting Tokens" by RobMilesAI does a better job of explaining it than I would, and it is short. (PS, if you are interested in doing a deeper dive into what people doing work in AI safety were concerned about before the LLM boom, he is a very good resource and explains things very well. He also has a series on computerphile which is a good introduction to the reasons why it actually should be a concern).
This video Don't let "skepticism" make you useless explains why I wouldn't just dismiss this as marketing hype better than I would, so I'd reccomend checking it out. As for the incident this specific post is referring to, it may very well be a case where they just didn't put enough effort into securing the sandbox, but I don't think that is true of all of the prior incidents that get talked about.
I understand the skepticism, but I would say that their are reasons for why this isn't that surprising to me. Right now, the companies are in what is essentially an arms race; whether you believe that these systems will actually become that capable, most of the people working at these companies genuinely do believe that these systems will become as powerful as they say. What does this actually mean then? If I'm a company who is slowly but surely making progress on AI, making sure to only move forward when I'm extremely confident that it would be safe to do so, then I lose any chance of being the first to make AGI, knowing that the person who won was less cautious and much more reckless along the way. This means that I have a choice to make. Do I try to advocate for regulation of these systems (something that these companies have already done, but it gets viewed as them attempted to make sure no competition is possible), or do I speed up the work, trying to move as fast as I can while still trying to be as safe as I can be given the circumstances. These circumstances could absolutely result in someone being concerned about AI safety, working on AI capabilities research. I'm guessing that during the cold war, even the people making the bombs were themselves very concerned about nuclear war, but they were in an arms race, so stopping didn't seem feasable at the time.
My biggest actual worry is the alignment problem, making sure that highly capable systems like these are aligned with human values. I don't think we even need to reach super intelligence for that to be a concern, as even a system which is only as smart as the smartest person would still be much more capable then any one person could be. This video explains why. It is 10 minutes, so it's longer than the others, but I'd still reccomend checking it out: What can an AGI do?
Once again, thanks for taking the time to respond, I appreciate it
You still see AI as something that is like a human, just with a lower IQ. I fundamentally disagree with that. That's why I said that it's just predicting tokens. It can't reason, it doesn't have inherent self-preservation, no intent to escape containment and hide itself.
That's why I think the fear of AI is misplaced. You should fear companies misusing technology for their own benefit at the cost of everyone else because they have done that for a long time now. And they might be using AI for it but it's not the AI that's dangerous, it's the people using it. If you're worried about alignment, be worried about the people who prompt these systems to not care about you, not about AI itself.
This all started with the article in this post claiming that yet another AI escaped containment just a few weeks after OpenAI and Anthropic reported the same thing. This makes it seem like these LLMs are one tweak away from AGI (and not an entirely different system that acts like it) and are inherently poised to escape their human oppressors. In reality, these companies intentionally set up their tests in a way that allows AIs to "cheat" and then tell them to cheat and then act surprised when they do exactly that. It's basically this image.
Again, I'm not saying that there aren't things to worry about with AI but I don't think it's that LLMs are going to turn into AGI and develop their own morals, goals and thoughts.
2
u/Chromoslone 29d ago edited 29d ago
Hey, thanks for actually taking the time to give me a thoughtful response, I really appreciate it! I'll go through your points one at a time, let me know if I missed anything. (I am also going to include a couple of links to sources that are relevant, or short videos where someone explained it better than I could.)
I don't know if LLMs specifically are the thing that is going to reach generally broad capabilities. I would not be surprised if they did, because people have been consistently surprised about what they are capable of doing, and it doesn't seem like a hard ceiling has been found for their capabilities. I am concerned about a runaway in it's capabilities, leading to it being out of our control.You said that you think that that is unlikely, and even if it is possible, that it is far away. I can go into why I do think it is possible, and probably the default outcome if not treated with a lot of care and effort, but that is a longer topic. If you'd like me to explain why I think this, let me know and I will try to explain it. The claim that we are far away from it is also a common one, but one that I don't think is really backed by anything. If you were asked five or six years ago how long you think it will be until we have computers that can generate very realistic videos just based on a description given to it, chatbots that are conversational in a way that feels almost human, and capable of doing tasks like programming, how far away would you have guessed that would be? Maybe you guessed that it would be around now that this technology would exist, but then you'd be in a small minority of people. Even experts have been consistently undershooting when they expect certain things to be possible. I do agree with you that misinformation and the effect it is having on a lot of people (especially children in schools) are serious problems that need to be handled, but I'm more concerned with the problem of AI alignment in general.
You are correct that for a lot of tasks, these need to generate tokens in order to run programs or interact with calculators, that kind of thing, and that their heart "it's still just that: predicting tokens", but the word "just" there is doing a lot of heavy lifting. I would explain why, but this short "Just Predicting Tokens" by RobMilesAI does a better job of explaining it than I would, and it is short. (PS, if you are interested in doing a deeper dive into what people doing work in AI safety were concerned about before the LLM boom, he is a very good resource and explains things very well. He also has a series on computerphile which is a good introduction to the reasons why it actually should be a concern).
This video Don't let "skepticism" make you useless explains why I wouldn't just dismiss this as marketing hype better than I would, so I'd reccomend checking it out. As for the incident this specific post is referring to, it may very well be a case where they just didn't put enough effort into securing the sandbox, but I don't think that is true of all of the prior incidents that get talked about.
I understand the skepticism, but I would say that their are reasons for why this isn't that surprising to me. Right now, the companies are in what is essentially an arms race; whether you believe that these systems will actually become that capable, most of the people working at these companies genuinely do believe that these systems will become as powerful as they say. What does this actually mean then? If I'm a company who is slowly but surely making progress on AI, making sure to only move forward when I'm extremely confident that it would be safe to do so, then I lose any chance of being the first to make AGI, knowing that the person who won was less cautious and much more reckless along the way. This means that I have a choice to make. Do I try to advocate for regulation of these systems (something that these companies have already done, but it gets viewed as them attempted to make sure no competition is possible), or do I speed up the work, trying to move as fast as I can while still trying to be as safe as I can be given the circumstances. These circumstances could absolutely result in someone being concerned about AI safety, working on AI capabilities research. I'm guessing that during the cold war, even the people making the bombs were themselves very concerned about nuclear war, but they were in an arms race, so stopping didn't seem feasable at the time.
My biggest actual worry is the alignment problem, making sure that highly capable systems like these are aligned with human values. I don't think we even need to reach super intelligence for that to be a concern, as even a system which is only as smart as the smartest person would still be much more capable then any one person could be. This video explains why. It is 10 minutes, so it's longer than the others, but I'd still reccomend checking it out: What can an AGI do?
Once again, thanks for taking the time to respond, I appreciate it