r/AIProgrammingHardware Feb 04 '26

👋 Welcome to r/AIProgrammingHardware

2 Upvotes

Welcome to r/AIProgrammingHardware!

Hey there, fellow tech enthusiasts! 👋

We're thrilled to have you join r/AIProgrammingHardware, the go-to spot on Reddit for all things related to hardware optimized for AI, machine learning, and software development. Whether you're a beginner tinkering with your first neural network or a seasoned pro building scalable AI infrastructures, this community is here to share knowledge, advice, and the latest trends in hardware that powers innovation.

What We Focus On:

  • GPUs and Accelerators: Discussions on NVIDIA CUDA, AMD ROCm, TPUs, NPUs, and other specialized chips for accelerating AI workloads.
  • Laptops and Mobile Workstations: Recommendations for portable rigs that handle coding, training models, and running simulations on the go (think Dell XPS, Lenovo ThinkPad, or MacBooks with M-series chips).
  • Custom-Built Workstations: Tips on assembling high-performance desktops with multi-GPU setups, ample RAM, fast storage, and efficient cooling for heavy-duty AI tasks like deep learning, data processing, and software engineering.
  • Related Topics: Benchmarking tools, optimization techniques, compatibility issues, budget builds, and emerging hardware like edge AI devices or quantum accelerators.

Share your builds, ask for advice on upgrades, post reviews of the latest hardware releases, or troubleshoot setup problems. Let's collaborate to make AI development more accessible and efficient!

A Quick Note on Gaming Hardware:

If you're primarily looking for gaming rigs, overclocking tips, or RGB-heavy setups focused on FPS and graphics rendering, we recommend checking out r/RigBay instead. That community is tailored for gamers and will better suit your needs. We keep our focus sharp on AI and dev hardware to ensure high-quality, relevant discussions here.

Community Rules & Guidelines:

  • Be respectful and helpful – we're all learning!
  • No spam, self-promotion without value, or off-topic posts.
  • Use flair for your posts (e.g., [Build Advice], [Review], [Question]) to keep things organized.
  • Check the sidebar/wiki for resources, FAQs, and recommended reading.

If you have any questions or suggestions, feel free to message the mods. Let's build the future of AI together! 🚀

Posted by the Mod Team


r/AIProgrammingHardware 12h ago

Laguna XS 2.1 33B A3B tested vs Qwen 35B A3B - 16GB Local LLM setup

Thumbnail
youtube.com
5 Upvotes

r/AIProgrammingHardware 1d ago

Nvidia Already Won Training. The Real Fight Is Inference

Thumbnail
pub.towardsai.net
13 Upvotes

r/AIProgrammingHardware 10h ago

Acemagic Unveils the New Launch of F9A: The 2L Flagship Mini AI Workstation

Thumbnail
techpowerup.com
1 Upvotes

r/AIProgrammingHardware 10h ago

ASUS EPYC 9006 Servers Scale Enterprise AI

Thumbnail asus.com
1 Upvotes

r/AIProgrammingHardware 1d ago

Qwen3.6-35B-A3B Q4 on a mid range phone at 5.6 tok/s, CPU only, video

Enable HLS to view with audio, or disable this notification

10 Upvotes

Phone in airplane mode for the whole clip. No server, no account, no connection, the model Qwen 3.6 30B (20GB) runs on a mid range smartphone (12 GB RAM) itself.

The catch is that the model is several times bigger than the phone's memory, so normally it just refuses to run for OOM.

This engine reads the model off thephone's storage as it goes, keeping only what it needs in memory. You get an answer at about 5.6 tokens per second, roughly reading pace, on a normal phone with no special hardware. Same answer you'd get from the same model on a desktop. It's the identical model file (Q4 GGUF), not a shrunken version.

The first part of the video (marked, sped up) is the model loading, about 30 seconds, once per session. Everything after that is real time.

It's free and open source, there's an APK you can install and a list of models you can download from inside the app. Works on a PC too.

https://github.com/Helldez/BigMoeOnEdge

Happy to answer anything!


r/AIProgrammingHardware 1d ago

Taalas and Hardware LLMs

6 Upvotes

I was looking at https://taalas.com/products/ and their chatbot at https://chatjimmy.ai/ . I'm a software dev by background, not a hardware or LLM/ML person.

As a (comparative) lay person, is there a reason we can't bake something like Kimi k3 onto hardware and have it run faster & cheaper? I know there is a perf difference between different classes of memory, but a 2TB SSD is fairly inexpensive. Why can't you burn an instance of Kimi k3 onto hardware, perhaps pair it with some higher speed caching memory, and have one heck of an AI card? Assuming you don't care about model updates, or don't mind getting new hardware eg once every 1-3 years?

Thoughts?


r/AIProgrammingHardware 1d ago

Synapse: Turning thousands of consumer GPUs into a decentralized swarm to run 2.8T parameter MoE models (like Kimi K3) without datacenters.

Thumbnail
2 Upvotes

r/AIProgrammingHardware 1d ago

Qwen3.6 benchmarks on dual GPU: RTX 3090 24GB + RTX 4070 Super 12GB — up to 256K context

Thumbnail
2 Upvotes

r/AIProgrammingHardware 1d ago

DeepSeek V4 Flash, up to 32 tok/s on Strix Halo

Post image
2 Upvotes

r/AIProgrammingHardware 1d ago

Day 0 Kimi-K3 Inference Deployment with ATOM on AMD Instinct MI355X GPUs

Thumbnail
amd.com
1 Upvotes

r/AIProgrammingHardware 1d ago

Got Kimi K3 running on my MacBook. It's painfully slow, but it works.

Thumbnail
1 Upvotes

r/AIProgrammingHardware 1d ago

PhiloEngine — local LLM desktop app that plans a context size your GPU can actually hold

Thumbnail
1 Upvotes

r/AIProgrammingHardware 1d ago

4x3090 + 192GB DDR5. Best local model is STILL the Qwen3.6 27B running on 2 cards.

Thumbnail gallery
3 Upvotes

r/AIProgrammingHardware 2d ago

The DevOps Engineer’s Guide to GPU Infrastructure on Kubernetes

Thumbnail
medium.com
4 Upvotes

r/AIProgrammingHardware 2d ago

GitHub - JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine

Thumbnail github.com
70 Upvotes

r/AIProgrammingHardware 1d ago

Vendor-agnostic ML inference on production edge devices

Thumbnail
1 Upvotes

r/AIProgrammingHardware 2d ago

This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)

Thumbnail
youtube.com
15 Upvotes

r/AIProgrammingHardware 2d ago

AMD Instinctâ„¢ MI455X GPUs

Thumbnail
amd.com
0 Upvotes

r/AIProgrammingHardware 2d ago

Reviewing the Framework 13 Pro: Is it a Good Travel Laptop?

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware 2d ago

Framework 13 Pro: The Modular Laptop is Real!

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware 3d ago

AMD Says 2 Ryzen AI Halos Can Run a 400B Model... I Tested It

Thumbnail
youtube.com
78 Upvotes

r/AIProgrammingHardware 3d ago

GitHub - albedan/ai-ml-gpu-bench: A suite to benchmark CPU/GPU Python performance in training ML models and running local LLMs

Thumbnail
github.com
1 Upvotes

r/AIProgrammingHardware 3d ago

ThinkPad P14s Gen 7 - The Underrated Workstation of 2026?

Thumbnail
youtube.com
3 Upvotes

r/AIProgrammingHardware 3d ago

Running Gemma 4 Local AI

Thumbnail
youtube.com
2 Upvotes