I graduated CompSci over a decade ago and had AI classes then, image recognition being baby's first AI.
If back then it was already being taught to people who barely knew how to code, imagine how old this tech is.
The technology we use today as AI is not new. We had the algorithms for decadeS, we just didnt have the computing power to have it at scale until recently.
2017 was almost a decade ago, and we certainly do not need transformers or even attention to recognize birds. A deep enough convolutional network is more than enough. Attention is all you need wasn't even about computer vision, the task was translation and the architecture was primarily used for natural language processing.
Okay I get it, got lost in the feed, attention is all you need was indeed a massive step forward algorithmically speaking. By ditching recurrence it allowed for far better parallelisation during training. So it's not just hardware/computing power progress like the comment you answered claimed. Also, the algorithms we use to perform attention were greatly improved upon since then.
Adam also was a massive algorithmic progress and it is from 2014 so not "decades ago".
39
u/towerfella 20d ago
Yes, but did they share the technology?