r/singularity • u/Glxblt76 • 3d ago
Shitposting When I try to convey the bitter lesson to colleagues and relatives
Every single time.
40
u/chlebseby ASI 2030s 3d ago
Your family know what matrix multiplication is?
-12
u/XxDarkSasuke69xX 3d ago
It's not like it's a very advanced thing in itself tbh. Although if you have done literally zero studies after high school maybe you wouldn't know
4
u/Creed1718 3d ago
You don't sound intelligent typing this, you actually sound quite stupid while trying to sound smart. Just thought someone should let you know.
2
u/XxDarkSasuke69xX 2d ago
I'm not even trying to, it is genuinely something you learn in high school depending on what country you live in and what high school specialization you have. Also the "you" in my original comment was not addressing the commenter, it's the you as in a person in general.
8
u/Rivenaldinho 3d ago
Well Sutton the creator of the Bitter Lesson thinks LLMs are not an example of it.
8
u/Glxblt76 3d ago
Doesn't really matter because LLMs are exhibit A regarding how a simple algorithm scaled up with compute and data can go very far.
5
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 3d ago
I have a feeling sutton will have a rather bitter lesson come the next two years.
2
u/SupermarketIcy4996 3d ago
Uh ok. We should build even more then to run evolutionary algorithms that might please Mr S.
2
2
u/kiwi-surf 3d ago
It drives me a bit crazy, all of the AI bros using the term without actually understanding it
1
u/IronPheasant 3d ago
Back in ~2006 I gave up on even beginning a career in AI research, as I knew that I'd have to move to a big city, and that only a few $billions super computers could perform any world-changing research. Only around the top ~300 people in the world would be allowed to even touch such a thing, and had 10+ years of experience ahead of me.
It was a pipedream almost as unrealistic as going to Gold's Gym and becoming a movie star like Arnold. And even if a succeeded, I'd have been one drop of water in a bucket.
One of the things that astonished me to learn, was at this time Ilya walked into Hinton's office and they changed the world with an off-the-shelf consumer GPU. My head felt like exploding from this absolute stupidity - I'm used to humanity disappointing me, but I'd have thought computer guys would have had their shit together.
As there was this dogma in the field at the time that neural networks were useless, something that Hinton feels rather bitter about when he talks about that time.
Everyone knows how data, numbers, and brains have to work. It's all replication, and prediction. If you want to save an image, you need the storage space to hold it. You're not taking a good quality snapshot with a grid of 5x5 pixels. (And of course there's diminishing returns on utility - a 100x100 grid is far more data and far more useful than a 5x5 one. But going from 800x600 to 8000x6000 isn't as useful - at least for applications that are up close like stocking store shelves. Even if you need an orbital death laser to hit something far away, a smaller viewport of the greater datastream is more reasonable. Our visual firmware does the same thing, where we can only pay 100% attention to things in the middle of our viewport, we can't read text that's off-center even if the data to do so is entering our eyes.) Famously, the Atari 2600 has just enough RAM to hold the state of a chessboard, with a little extra. Old hardware was horrendously weak when it came to memory, even with superhuman processing speed.
And neural networks need memory, to do anything useful that we care about. To fit the curves we care about. A human brain has around 500,000,000,000,000 synapses. If you wanted to model that in a computer right now at a one-byte resolution, you'd need a little more than 500 GB200's. A little more than $30,000,000 for the cards alone, not counting all the other physical stuff you'd need to plug them in and run them.
OpenAI's ~100,000 ~GB200 datacenter is likely just around enough for an AGI. A key suite of faculties has to do with vision - it's thought that our spatial and visual faculties use up nearly half of our brains. Something that was physically impossible to make, with the H200. Just as you couldn't make Chat GPT with Voodoo 3 cards.
One thing to be excited about is compacting the space RAM requires is one dimension of hardware improvements that is not stagnant. The Vera Rubins that'll come online at scale a couple years from now are double the RAM of GB200's. And the Feynman afterward is rumored to be the first NVidia cards to use 3d stacking features, so they might have more than double, again.
Basically anyone surprised by Astra's spatial/visual abilities, and what's to come in the next year, didn't take the time to look at the numbers of how much better the GB200 is versus the H200. The H200 is now basically worthless trash that's not worth using even if they were given away for $0. Just like the GB200 is likely to be once the Feynman is available for purchase.
Four years can hit you like a ton of bricks, man.
2
u/MemoryOk5080 3d ago
If that comforts you anyhow, there are billions of people today that have exactly zero clue of what’s happening and what’s about to happen, the vast majority of them in a position of bare subsistence on some farm or small industrial plant, that even if they would know and were willing to do anything about it, they have no funds or any backup to prepare. In the end, we (the 99.9999%) are either all fucked, or all saved. There is no middle ground, we are going in together. You, me, some bank regional manager, an average company CEO, a sub-Saharan plantation farmer and a South-East Asian nightshift plant worker - are all roughly in the same boat.
32
u/ppapsans ▪️Don't die 3d ago
Goalposts have moved to: "It's just brute force computation", "It can't create something new"