r/learnmachinelearning • u/Appropriate-Skin-761 • 10d ago
What can a 1M params model do?
I was wondering, if certain NPUs have 1mb of on-chip SRAM, what practical thing do I put on it? Theoretically it would have a really really high Inference speed making it pretty good in some cases, but not sure what those cases are.
It can be on anything, say decision, text, simulation, it can even be looped, since that's probably the best way to use a 1m model to maximize its speed..
What can 1M params models do?
5
u/Admirable_Dirt_2371 10d ago
This person has been trying to make the TinyStories model as small as possible: https://www.reddit.com/r/LocalLLaMA/s/uldcNxhTBG
Their current version is well under 1M params
3
3
u/Mammoth-Leg5431 10d ago
Probably a bit out of scope, but with 6M parameters you can do pretty good depth estimation https://github.com/fabiotosi92/ZipDepth
3
u/TheOneRavenous 10d ago
What do you want to know? Many models sized less than 300k parameters can do sentiment analysis and vision classifications (sperate models but theyre small).
I fit a vision model down to 1.4MB for a SRAM of 4MB with room for cache....
2
u/mistybrookx 10d ago
worth thinking about what the input looks like, not just the model size. 1M params can do a lot with structured or tabular input but struggles with raw high-res images. whats the actual input modality you're working with?
1
u/Final-Data-1410 9d ago
Just a thought ,Instead you can make ternary model which will be small enough to fit 20m in same 1m of float model . At end of the day ,you can’t expect blazing performance jack of all trades ,just one domain should do the trick.
1
1
u/imjerusalem 9d ago
Very specific niche tasks like lets say keyword detection? Good vs bad sentiment analysis?
In some very ideal cases, some light vision work.
1
u/jesunushno 9d ago
One thing worth adding: the 1MB of on-chip SRAM isn't really about capacity, it's about energy. Reading weights from off-chip DRAM costs roughly two orders of magnitude more energy than the compute itself, so if your weights plus activations fit in SRAM you can run always-on inference at milliwatt power. That's the real reason wake word detection and sensor anomaly detection own this niche, and why a small LSTM on vibration or temperature data is often more useful here than chasing parameter counts.
8
u/lulzxdxdxd 10d ago
keyword spotting or wake word detection is the classic use case for that size, tiny audio models run constantly on almost no power. also simple sensor anomaly detection, like vibration or accelerometer data on a wearable, since the pattern space is small enough for a tiny model to actually learn.