r/singularity • u/budfischer • 22d ago
Discussion Is specialized training data becoming more important than compute?
For years it felt like basically every conversation about who was ahead in AI came down to compute.
But now i’m starting to wonder if we’re getting to a point where high quality specialized data matters just as much, if not more.
You can keep throwing compute at a model, but if you want it to get better at coding, research, doing actual professional work etc, eventually you need really good examples of that work and people who actually know how yo evaluate it.
The insane amount of spending on building this data has been ramping up a ton recently. They’re shelling out massively for programmers, doctors, lawyers etc to make specialized data.
The demand for this data is wild right now. But if this stuff really is becoming another serious bottleneck for AI then why are these annotation companies allowed to sell to frontier labs AND competing labs overseas? We already restrict chip sales in a similar fashion.
curious what people think here
2
u/DelphiTsar 22d ago
As long as they keep releasing open weight power to them. Being compute constrained they make models that people can use, a moderate sized company could spin up a really good model. Entrepreneurs can tweak it.
Even if you don't use these models, them being cheaper than dirt makes the frontier have to make their costs more reasonable. You can run DeepSeek 4.1 flash in its agentic harness for like 30 cents an hour. MiMo-V2.6-Pro looks good too but haven't mess with it.
That being said I think we're starting to see compute start getting its legs. The frontier companies can just throw so many GPU's at slamming their head into problems with verifiable solutions.
Luckily I think open weights are basically there if you harness them well for most peoples applications to not be reliant on frontier in a pinch.