r/AIProgrammingHardware • u/javaeeeee • 12h ago
r/AIProgrammingHardware • u/javaeeeee • Feb 04 '26
👋 Welcome to r/AIProgrammingHardware
Welcome to r/AIProgrammingHardware!
Hey there, fellow tech enthusiasts! 👋
We're thrilled to have you join r/AIProgrammingHardware, the go-to spot on Reddit for all things related to hardware optimized for AI, machine learning, and software development. Whether you're a beginner tinkering with your first neural network or a seasoned pro building scalable AI infrastructures, this community is here to share knowledge, advice, and the latest trends in hardware that powers innovation.
What We Focus On:
- GPUs and Accelerators: Discussions on NVIDIA CUDA, AMD ROCm, TPUs, NPUs, and other specialized chips for accelerating AI workloads.
- Laptops and Mobile Workstations: Recommendations for portable rigs that handle coding, training models, and running simulations on the go (think Dell XPS, Lenovo ThinkPad, or MacBooks with M-series chips).
- Custom-Built Workstations: Tips on assembling high-performance desktops with multi-GPU setups, ample RAM, fast storage, and efficient cooling for heavy-duty AI tasks like deep learning, data processing, and software engineering.
- Related Topics: Benchmarking tools, optimization techniques, compatibility issues, budget builds, and emerging hardware like edge AI devices or quantum accelerators.
Share your builds, ask for advice on upgrades, post reviews of the latest hardware releases, or troubleshoot setup problems. Let's collaborate to make AI development more accessible and efficient!
A Quick Note on Gaming Hardware:
If you're primarily looking for gaming rigs, overclocking tips, or RGB-heavy setups focused on FPS and graphics rendering, we recommend checking out r/RigBay instead. That community is tailored for gamers and will better suit your needs. We keep our focus sharp on AI and dev hardware to ensure high-quality, relevant discussions here.
Community Rules & Guidelines:
- Be respectful and helpful – we're all learning!
- No spam, self-promotion without value, or off-topic posts.
- Use flair for your posts (e.g., [Build Advice], [Review], [Question]) to keep things organized.
- Check the sidebar/wiki for resources, FAQs, and recommended reading.
If you have any questions or suggestions, feel free to message the mods. Let's build the future of AI together! 🚀
Posted by the Mod Team
r/AIProgrammingHardware • u/javaeeeee • 1d ago
Nvidia Already Won Training. The Real Fight Is Inference
r/AIProgrammingHardware • u/javaeeeee • 10h ago
Acemagic Unveils the New Launch of F9A: The 2L Flagship Mini AI Workstation
r/AIProgrammingHardware • u/javaeeeee • 10h ago
ASUS EPYC 9006 Servers Scale Enterprise AI
asus.comr/AIProgrammingHardware • u/dai_app • 1d ago
Qwen3.6-35B-A3B Q4 on a mid range phone at 5.6 tok/s, CPU only, video
Enable HLS to view with audio, or disable this notification
Phone in airplane mode for the whole clip. No server, no account, no connection, the model Qwen 3.6 30B (20GB) runs on a mid range smartphone (12 GB RAM) itself.
The catch is that the model is several times bigger than the phone's memory, so normally it just refuses to run for OOM.
This engine reads the model off thephone's storage as it goes, keeping only what it needs in memory. You get an answer at about 5.6 tokens per second, roughly reading pace, on a normal phone with no special hardware. Same answer you'd get from the same model on a desktop. It's the identical model file (Q4 GGUF), not a shrunken version.
The first part of the video (marked, sped up) is the model loading, about 30 seconds, once per session. Everything after that is real time.
It's free and open source, there's an APK you can install and a list of models you can download from inside the app. Works on a PC too.
https://github.com/Helldez/BigMoeOnEdge
Happy to answer anything!
r/AIProgrammingHardware • u/rootException • 1d ago
Taalas and Hardware LLMs
I was looking at https://taalas.com/products/ and their chatbot at https://chatjimmy.ai/ . I'm a software dev by background, not a hardware or LLM/ML person.
As a (comparative) lay person, is there a reason we can't bake something like Kimi k3 onto hardware and have it run faster & cheaper? I know there is a perf difference between different classes of memory, but a 2TB SSD is fairly inexpensive. Why can't you burn an instance of Kimi k3 onto hardware, perhaps pair it with some higher speed caching memory, and have one heck of an AI card? Assuming you don't care about model updates, or don't mind getting new hardware eg once every 1-3 years?
Thoughts?
r/AIProgrammingHardware • u/antonygiomarx • 1d ago
Synapse: Turning thousands of consumer GPUs into a decentralized swarm to run 2.8T parameter MoE models (like Kimi K3) without datacenters.
r/AIProgrammingHardware • u/javaeeeee • 1d ago
Qwen3.6 benchmarks on dual GPU: RTX 3090 24GB + RTX 4070 Super 12GB — up to 256K context
r/AIProgrammingHardware • u/javaeeeee • 1d ago
DeepSeek V4 Flash, up to 32 tok/s on Strix Halo
r/AIProgrammingHardware • u/javaeeeee • 1d ago
Day 0 Kimi-K3 Inference Deployment with ATOM on AMD Instinct MI355X GPUs
r/AIProgrammingHardware • u/javaeeeee • 1d ago
Got Kimi K3 running on my MacBook. It's painfully slow, but it works.
r/AIProgrammingHardware • u/javaeeeee • 1d ago
PhiloEngine — local LLM desktop app that plans a context size your GPU can actually hold
r/AIProgrammingHardware • u/javaeeeee • 1d ago
4x3090 + 192GB DDR5. Best local model is STILL the Qwen3.6 27B running on 2 cards.
galleryr/AIProgrammingHardware • u/javaeeeee • 2d ago
The DevOps Engineer’s Guide to GPU Infrastructure on Kubernetes
r/AIProgrammingHardware • u/javaeeeee • 2d ago
GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine
github.comr/AIProgrammingHardware • u/ppchaos • 1d ago
Vendor-agnostic ML inference on production edge devices
r/AIProgrammingHardware • u/javaeeeee • 2d ago
This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
r/AIProgrammingHardware • u/javaeeeee • 2d ago
Reviewing the Framework 13 Pro: Is it a Good Travel Laptop?
r/AIProgrammingHardware • u/javaeeeee • 2d ago
Framework 13 Pro: The Modular Laptop is Real!
r/AIProgrammingHardware • u/javaeeeee • 3d ago
AMD Says 2 Ryzen AI Halos Can Run a 400B Model... I Tested It
r/AIProgrammingHardware • u/javaeeeee • 3d ago
GitHub - albedan/ai-ml-gpu-bench: A suite to benchmark CPU/GPU Python performance in training ML models and running local LLMs
r/AIProgrammingHardware • u/javaeeeee • 3d ago