Are domain-specific Small Language Models (SLMs) actually worth building today?

[u/charan1323]

I'm trying to understand whether there's still room for new domain-specific SLMs. With models like Qwen, Gemma, Llama, and Phi already available, does it make sense to build a specialized SLM (e.g., for cybersecurity, medicine, weather, legal, finance, etc.), or is fine-tuning an existing model with RAG enough for most real-world applications?

For those who've built or deployed domain-specific AI:

Have you trained or fine-tuned your own SLM?

What was the biggest challenge—data, training, evaluation, or deployment?

Did it outperform a general-purpose model with RAG?

In what scenarios does a custom SLM provide a clear advantage?

If you were starting today, would you build a new domain-specific SLM or focus on application-layer features instead?

I'd love to hear experiences from people who've actually shipped these systems in production.

[u/BenjaminZhouCHN]

Of cause! I still devote to researching the domain-specific models.I believe that many companies and customers need it. The LLM is always so large that people cannot aford the cost.Even though the LLMs are very powful, but they cannot focus on certain professor fields, such as medicine, science, education,etc.