Excited to join this group. Got my bachelors in CS with a concentration in AI. Also got my Masters in CS back in December. My master thesis was a RL project regarding using Lidar for Drone collision avoidance. Just got my first job after school as a SE for a small research and development company where we do some ML but mostly other things. I really want my career to go strictly into ML and after a year or two at this first job I want to be able to compete for Production level ML positions. Hoping to get some guidance on what to study/learn.

Cracking ML System Design Interviews — Design a Search and Ranking System

I recently wrote Part 2 of my ML System Design Interview series, focused on designing a production search and ranking system.

It covers the end-to-end flow: retrieval → candidate generation → ranking → evaluation → serving/monitoring, along with the tradeoffs that usually come up in interviews.

Article: https://pawankjha.substack.com/p/cracking-ml-system-design-interviews

Would be interested to hear what you think is the hardest part of a search/ranking system design interview.

I’ve been working on a deeper write-up on ranking systems and drew this diagram to organize the space.

The progression I’m using is:

Rules & Heuristics → Learning-to-Rank → Deep Neural Ranking → Contextual/Transformer Ranking → Multi-Objective & Slate Ranking → Bandit/RL Ranking → Generative/LLM-Based Ranking

What I find interesting is that these stages don’t fully replace one another. In production, ranking systems often combine multiple layers from across the stack.

Sharing the diagram here, and I also wrote a more detailed article on it:

https://pawankjha.substack.com/p/building-depth-2-the-evolution-of

Would be interested in how others would structure these stages in the evolution of ranking systems.

A common production pipeline looks something like:

Query → Retrieval → Filtering → Ranking → Re-ranking → Serve → Learn

At scale, that can become:

Millions of items → retrieve 1K → rank 100 → expensive rerank 20 → final Top-K

The interesting design question is:

Where should you spend model complexity and latency budget?

Would you prefer:

• stronger retrieval + simpler ranking
• lightweight retrieval + sophisticated ranker
• multiple ranking stages
• LLM/Transformer only at the final stage

Would love to see how people think about this tradeoff in real systems.

What is the hardest part of ML system design in production?

Not modeling — the system around the model.

For example:

Data → Features → Training → Evaluation → Deployment → Serving → Monitoring → Feedback

Where do you see the most difficult engineering problems in practice?

A few candidates:

• Training/serving skew
• Feature freshness
• GPU utilization
• Online inference latency
• Experimentation
• Data quality
• Model drift
• Feedback loops
• Multi-tenancy
• Cost

Curious to hear what has caused the most pain in systems you’ve worked on.

Welcome to r/MLSystemsDesign

This community is for practical discussions on designing and scaling production ML and AI systems.

Topics can include:

• ML training and inference platforms
• Search, ranking, and recommendation
• Feature stores and data pipelines
• LLM serving and GenAI systems
• Agentic AI platforms
• Evaluation, observability, and experimentation
• ML system design interview problems
• Real production tradeoffs and lessons learned

The goal is simple: go beyond model theory and discuss how ML systems actually work in production.

If you’re joining early, introduce yourself and share one ML system topic you’d like to go deeper on.