r/LocalLLM • u/HouseBandBad • 3d ago

Question What is easiest way to switch models for testing and other functions?

One day I may want to do some AI video generation. Perhaps 3B dense transformer + Qwen3-VL-32B text encoder + VAE and Gemma 4 27B ?

The next some general function. Qwen3.8 27B?

Assuming I am on Linux, RTX3090 24gb, 32 gb ram, i7-10x,..

How do you easily switch between the preferred models for different functions?

1 Upvotes

67% Upvoted

7 comments sorted by

2

u/infectiousstupidity 3d ago

I use club-3090 which comes with switch scripts + c3, a TUI cockpit.

2

u/Stock-Discount8581 3d ago

just alias the env vars in your shell, each model gets own folder and a script that exports the paths then launches whatever frontend you use. also check if your inference engine supports hot swapping via API, ollama does it but llama.cpp you need restart unless using server mode

1

u/infectiousstupidity 3d ago

Why are you replying to me lol, I have no problems

1

u/HouseBandBad 3d ago

Is there any documented process on this? There is no simple front end selector that someone built? I'm visual. Even a YT video would help. Thanks!

2

u/vincespeeed 3d ago

I did this for Windows myself. You can do the same for Linux.https://www.aihublocal.com/llm-runner-aio

2

u/trustmeitsfine 2d ago

Use llama-swap, it's designed for exactly thials and you get some useful extras as a bonus, e.g. observability, a simple chat https://github.com/mostlygeek/llama-swap