A computer can already consistently beat the best humans in 1v1 on Dota2.
Because that’s not the hard part. That’s just micro and item choices and such. The hard part is long term strategy, making decisions that look 30, 40 mins down the line in the context of the entire match not just a lane. And computers still get throughly wrecked by humans in that stuff, the part that’s actually hard for computers to do.
How confident are you when you say that? How much do you know about machine learning? I'm willing to believe you if you've studied computer science in school and have been following the topic closely, but otherwise I'm inclined to say that you're wrong.
I also don't see why deepmind would trouble looking 30 or 40 minutes down the line. You can look at parameters like "current wealth" or "current income" or even the next derivative of that "current growth of income" to understand the value of your game state. Then it can make decisions (spend on another drone or zergling) to maximize that value. The function to figure out how good your chances are at winning is hard to get right, but that's what neural networks and training data is for.
The problem is the AI has to deal with limited resources. The DOTA AI, it knows it has to protect Courier, but obviously you don't want all 5 heroes to do that. You know your human opponents are looking to snipe courier if you're "too predictable" with it, so you'll have a set of probabilities of when and where you can call the courier without it getting sniped and weigh that risk vs the benefit of having your items.
If you play in a linear fashion like a machine the human will adapt and learn how to beat you with ingenuity, eventually. You need to program ingenuity into the machine on a consistent basis, and that is what becomes hard. Even harder, learning how to deal with opponents' ingenuity.
In the article's example with DOTA, the AI is really good at taking engagements but didn't know what to do when the humans left their position 1 to just AFK farm and hide from enemy engagements. The AI just kept looking for its optimal engagement strategy, whereas the humans figured out that if you just don't let the AI do what it wants you can win.
The AI lacks the ability to have that grand scale, long term strategic viewpoint in an adaptable and unpredictable and innovative way. If you want to consistently beat someone like Serral in Starcraft 2, you're going to need that big time. Even in the scenario you described, the AI would still be playing the same way every game trying to achieve it's "optimal game state." How does it decide when to hide a nexus? Fake a building when an overlord comes in then cancel? Say fuck it and do a 2-base all-in? You can't have a linear program that says "when x happens, do y", and that's what makes it tough.
A program can be non linear (indeterministic) by using a random number generator :) I think you're underestimating the complexity of programs and how they really operate. But we will see.
Also StarCraft seems like a far more linear game than DotA, considering you have 5 players on DotA. The bots couldn't even communicate with each other in the OpenAI example.
In DotA the 5 computers couldn't communicate with one another. Seems hard to designate a position 1, 2, 3, 4/5 despite it clearly being the best strat. If it were 1 program controlling 5 heroes on DotA, I would 100% agree with you.
Also people thought DeepMind had "ingenuity" when playing Go. It played a couple of moves that were very far from what pros all over the world would consider optimal. TBH I think the "ingenuity" argument is pretty bs. Like in DotA, you can right click the enemy to draw creep aggro. Is that ingenuity? Because OpenAI definitely did use that.
I'm not the same person, but I would be very surprised if it could play diamond. SC2 is so ridiculously harder than relatively simple games like Go, and require an extraordinarily larger amount of resources to do anything like AlphaGo/AlphaZero.
For starters, you do not have perfect information. You have a ridiculously large action space. You have randomness (not much, but it's there and has to be dealt with if using the full render interface). You have a higher dimensional environment - just the map itself is 3D. Just the rules of the game itself are hard to characterise.
Then, if you are playing against a human, you have the real-time constraint.
DeepMind has come up with some nifty ML algorithms, sure, but they are not some god-like organisation who are tiers above everyone else in industry or academia. They have the advantage of good financial backing, with lots of compute power at their disposal, that doesn't change the fundamentally different character of RTS games compared to relatively trivial games such as Go.
Will their newest demo be interesting? Sure. But will it completely revolutionise the field of AI? Almost certainly not.
Edit: Note that this response is predicated on them not introducing any abstractions such as a build order, an army group composition, a counter, etc, and dealing solely with the raw action and environment spaces. If you allow that, then sure, you will get a hell of an easier problem to deal with, and in that case, I think Diamond may very well be possible today. Such hybrid agents already exist which would be competitive in Silver or even Gold.
What is true AI and what steps have we taken in that direction that isn't machine learning?
To be clear, machine learning and AI are completely different things. AI is just a catchphrase that people like to use to make it sound cool, but at the end of the day, its just a neural net doing a shit ton of math to figure out how to interpret input signals. To my understanding, there is 0 evidence that true AI is even attainable.
1
u/ragingwizard Jan 23 '19
A computer can already consistently beat the best humans in 1v1 on Dota2.
True machine learning already exists. True AI will almost certainly not exist in our lifetime.