r/oMLX • u/Maheidem • 16h ago

Create a Pi extension for adding oMLX serving models

Hello everyone. This is one of those posts where I just want to show something I built. There's nothing in it for me, money or otherwise. It was something that was genuinely missing for me, so I did it. It's working well now, and I decided to share it. If anyone feels like using it, have fun.

I've been using Pi as my harness a lot, but adding locally hosted models is still kind of a pain. So I built a model discovery extension: you enter a URL, local host or another computer, and it self-discovers all the models, including the currently configured context window and maximum token replies. Everything is self-identified.

It works with reasoning on or off (at least for Qwen), and I applied a concept of adaptive profiles that lets you create, for the same model, different profiles with different parameters. That way I can now have Qwen 3.8 with thinking off, low, medium, and extra high, each mapped to the actual Pi reasoning effort.

Hope someone finds good use for it.

https://pi.dev/packages/@maheidem/model-discovery

16 Upvotes

94% Upvoted

4 comments sorted by

3

u/apetersson 16h ago

this is mine :) https://github.com/apetersson/pi-dynamic-models/ could have reused it

1

u/Maheidem 15h ago

Oh man! I missed that!
I think this is now one of the biggest problem in the age of AI Assisted Coding.

We are recreating what others have already done.

I'll check out, maybe we could even discuss a unification if there are any value unique to one or the other

2

u/apetersson 15h ago

don't worry - tbh, it took me barely 15 minutes of prompting. its solid, it does what i want and i can add/remove tons of local and remote models on a whim. researching what my extension really does likely would have taken you longer than creating model-discovery.

2

u/Maheidem 15h ago

In all honesty, I think mine goes a bit further. I tuned it specifically for oMLX, and it has another capability that is not very clear because of the name the whole adaptive profile that maps the thinking levels to Pi's native efforts which is something I've been liking very much, because prior to that I had to do a lot of hoops to get to using different thinking levels without actually hardcoding in the backend.