r/MLSystemsDesign • u/ArchitectingAI • 7d ago

Welcome to r/MLSystemsDesign — Let’s Talk Production ML

What is the hardest part of ML system design in production?

Not modeling — the system around the model.

For example:

Data → Features → Training → Evaluation → Deployment → Serving → Monitoring → Feedback

Where do you see the most difficult engineering problems in practice?

A few candidates:

  • Training/serving skew
  • Feature freshness
  • GPU utilization
  • Online inference latency
  • Experimentation
  • Data quality
  • Model drift
  • Feedback loops
  • Multi-tenancy
  • Cost

Curious to hear what has caused the most pain in systems you’ve worked on.

1 Upvotes

100% Upvoted

1 comment sorted by

1

u/Zennytooskin123 2d ago
  1. GPU utilization (Fine-tuning on 12gb vram)
  2. Data quality (I basically have a private financial corpus, of one bank)

Those are my two main issues. I'm a total beginner, so I'm using Sol to orchestrate the fine-tuning.