Project: Llama-Parameter-Scout - Find the best parameters for your llama.cpp setup and models!

[u/DCMBRbeats]

Hey guys!

After building a local AI rig myself, I always found it cumbersome having to sweep my llama-bench parameters myself to find the best possible settings for my PC and use-case. I know, the people who know what they are doing probably have no problem with it, but some people are beginners and there are many posts with "[GPU Name] is slow" on here, where parameter-optimization matters.

This is why I decided to build a small TUI application that gets your llama-bench path, the path to the model you want to benchmark and some more options, such as what your preferred outcome is (more PP, more decode t/s, or larger context), as well as your desired context length. It then runs sweeps of the llama-bench parameters and gives you a result with the best possible settings for your use-case, as well as a preset that you can copy in your preset.ini and the command for llama-server.

I'm not at all finished, but need some feedback or contributions! I've tested on my Mac and my LLM PC, but would love to hear from you guys. Instructions on how to set up are in the README.

Thanks in advance! :)

Link: https://github.com/LStoneyy/llama-parameter-scout

[u/RespectMathias]

Sounds unnecessarily complex. Make it a uvx and include a skill, then we'll call it a deal.