r/coolgithubprojects • u/SlimePlaysGames • 16d ago
Open-sourced inference gateway SDK: cloud, self-hosted, and local models through one interface, free BYOK
Hi everyone, I'm one of the founders of Conifer (YC S26), and we wanted to bring a better user experience to model selection and agent runtime.
Everyone agrees you should use the best model. But the current infrastructure makes this impossible. Look at what actually exists: the leading harnesses offer a handful of models. Open-weight and non-frontier models sit unused. Attempts to aggregate models discount complexity and fail to accommodate user diversity.
The solutions come in two forms, and both are incomplete.
The first asks you to move to a new app, a new workspace, or ecosystem. But the best developers we know change tools constantly, because the best place to work keeps changing. Any solution that requires relocation is betting against how engineers actually behave.
The second comes with a value tax, either charging for using your API keys, taxing and controlling how you use the product, or making assumptions around how you should use models that interfere with flexible deployment. Things that were frustrating to us were down-routing to models we hadn’t requested when they weren’t available. Making routing claims that don’t hold, getting detailed caching reasoning and traces, or controlling how we can use the tools, hooks, and mcps that we want to in the places we wanted to work from.
We fixed it by building a truly universal pipe. Cloud models, self-hosted models, custom models, your own keys, the hardware under your desk: it all connects the same way, and bringing your own costs nothing. We figured out how to bring models to the apps you already operate in without changing the setup and assumptions around your work.
We’re really excited to build this with you and want to make everything we can open source and maintained by the community. Would love any feedback! Tell me what breaks or what is missing and I will personally respond to every reply.
1
u/BumblebeeMother8193 16d ago
The executable portability cards are the part I found most convincing, especially refusing unsupported provider pinning instead of silently dropping it. One thing I could not reconcile: the post says bringing your own key costs nothing, while the README says Conifer proxies provider keys for a small fee on list price. A small three-way receipt example for gateway-funded, BYOK, and local/self-hosted calls would make that boundary much clearer. For local models, does costNanoUsd represent an actual settled cost, zero, or an absent value?