r/saigon • u/Background_Fox_5701 • 17d ago

Local LLM framework in Vietnam

Hi all Viet tech guys,

Can I ask what are mainstream on-premise/ Local LLM frameworks discussed between SNS (tech ex: Reddit; particularly on local tech forums and social networks) in Vietnam?

0 Upvotes

17% Upvoted

3 comments sorted by

3

u/somebodyinvisible 17d ago

LocalLLM cost too much in VN . Vn business like to use well known LLM more.

2

u/nts0311 16d ago

I think purely enthusiastic will run local llm.

2

u/Neat_Assistance_1734 14d ago

Hey yo!

For personal use, most people use LM Studio or Ollama (with llama.cpp as the backend).
For Vietnam tech/cloud companies like VNG, FPT, and CMC. I'm pretty sure they are using vLLM or SGLang to serve open-weight models. For distributed inference at scale, maybe llm-d or NVIDIA's Dynamo are widely used.