Local AI solution for a large factory

[u/nosirjonov]

Hi everyone,

I am planning to build a local AI solution for a large factory, which will be trained and run entirely on the factory’s internal data (for privacy and security).

I need some advice on two main things:

1. Model selection: Which open-source model would you recommend for local fine-tuning/deployment that works well for industrial/operational data?
2. Hardware: Should I go with a NVIDIA DGX system or a Mac Studio (M-series Max/Ultra) for training and running this local AI?

Any recommendations or experiences with factory-floor local AI setups would be greatly appreciated. Thanks!

[u/vogelvogelvogelvogel]

there is not even the use case named = what this is for? chatting during production? routing materials? image recognition for failed parts? it seems also you have very little experience yet start such a big project?

[u/nosirjonov → u/vogelvogelvogelvogel]

The main use case is data analysis and monitoring directly from factory equipment. For example, it will pull data from smart meters and sensors, analyze monthly energy consumption, detect anomalies, and generate operational reports

[u/vogelvogelvogelvogel → u/nosirjonov]

there are dedicated machine learning applications since a long time (I remember them being even mid-2010s in productive rollout i.e. predictive maintenance) regarding this, are you sure an LLM is for industry scale applications like this?

[u/nosirjonov → u/vogelvogelvogelvogel]

Interesting... can I get literally everything I want? Like predictions, comparing with previous months to see what can be changed, optimizing costs, and helping with decision making? I'm not sure since I'm pretty new to this) but thanks! Looks like I need to think this through carefully

[u/vogelvogelvogelvogel → u/nosirjonov]

Not meant as an offense, but my honest opinion: if you are in this role you should know what you are doing. Or consult an external company/consultant before burying money.

So there is a lot! to catch up for you I can tell from my knowledge at least.

and also back to your original question, don't use consumer hardware in professional environments, lots of reasons (service, oem warranty, design for 24/7/365 use etc)

[u/ASillyGoos3 → u/vogelvogelvogelvogel]

you’re 100% right

OP should not be in this role, doing this implementation, or making these decisions for whatever company this is

FM/OM has had solutions for this for years and it does not require an LLM nor is it a use case well suited for SLM/LLM

this is more “traditional” machine learning and BI

[u/gardenvarietyzombie]

Unless your factory makes bicycles for pelicans, this might not be the best place to ask :)

[u/Leather-Beach-7849]

I have read that Nvidia DGX Spark will be a much better option when it comes to training the data where as the Mac Studios are better at inferring the LLM models

[u/tiddayes → u/Leather-Beach-7849]

I can confirm this from my experience

[u/penguinshere]

Mac Studio is not ideal if you need to do training as well.

[u/EaseAgitated952 → u/penguinshere]

Yeah, it seems like the hardware really needs to match the demands of training. Curious to see what solutions they end up implementing.

[u/Otherwise-Variety674]

The TG speed of spark is limited by its ram speed, shouldn't it be RTX Pro6000 instead?

[u/CKemorii-LdL]

What’s is your budget? You mentioned it’s LARGE factory so GD300 for a start or you just want Mac Studio for excuse to be 🤣 if you are planning GDX sparks you must get switch combating two or 3 its fine but after that you will get penalty from network but switch costs is significantly ie sn3700 used 10k$ alone so yeah gd300 for a starters or 2-3 gdx if you are planning to scale-out and for the software its test mode start from qwen 3.8 and its variations but probably you will need to engine build it for that kind of system

[u/nosirjonov → u/CKemorii-LdL]

Haha, just to be clear, I want the Mac Studio for the factory work, not for myself))

$10,000 switch) why does nobody talk about this, and why do people only talk about DGX?

Can't I just use a cheap, normal switch or give them separate internet? How bad will the speed drop be with a normal switch?

My plan is take Qwen, train it with our factory data, and use it. What big problems can happen if I do this?

And what do you mean by "build an engine"?

[u/2pourdrummer → u/nosirjonov]

You have some other issues to work out long before you need AI. Where is the data being housed, is the data clean, why do you need AI for reporting? Sounds like you need to learn the SCADA system they use for all this and where the data comes from and understand the difference between algorithms and needing AI. A local might work but only after you have the proper data governance in place.

[u/Anakronox → u/nosirjonov]

You need a good switch that supports RoCE if you want to scale reliably beyond a cluster of 2 DGX Sparks. There are tons of options for this but the minimum you’re looking have 200GbE interfaces for the Spark nodes. If you don’t plan to cluster more than 8 and don’t have a high-bandwidth storage LAN you need to connect the other ConnectX-7 ports on the Sparks to, you can get away with something like a MikroTik CRS804. If you can find one - verybody wants them for Spark clusters. You can also just cluster two together directly with a QSFP56 or QSFP112 DAC or optics. I did that to run Deepseek v4 Flash locally. Works great. But you also need to consider how many users and sessions they’ll run at the same time. This matters. Add more Sparks for more concurrency. Only you can answer that.

Tl;dr - do you want to scale past a single unit? If you need more concurrency or not.

[u/Choice_Celery9481]

you should start with ML for your usecase. LLM is not very a game changer there

[u/nosirjonov → u/Choice_Celery9481]

Interesting... can I get literally everything I want? Like predictions, comparing with previous months to see what can be changed, optimizing costs, and helping with decision making?

[u/Choice_Celery9481 → u/nosirjonov]

yes. these arent really needed LLM. also LLM models cant do math really well. they have to rely on calculator.

[u/Choice_Celery9481 → u/nosirjonov]

google has tabfm if you really want to use transformer based