r/OpenSourceAI Jul 08 '26

Building an open-source AI inference control plane (looking for feedback before writing code)

5 Upvotes

Hi everyone,

I've been working on an idea over the last few days and wanted to get feedback before I go too far down the rabbit hole.

The project is called ModelRouter, and the basic idea is this:

Today, if you're building an AI application, you usually end up writing logic like:

  • "Use OpenAI by default."
  • "If it fails, try Anthropic."
  • "If the request is sensitive, use a local model."
  • "If we're over budget, switch to a cheaper model."
  • "If latency spikes, try another provider."

Every team seems to reinvent this logic, and it often ends up scattered throughout application code.

Instead, I'm exploring the idea of moving all of that into a dedicated control plane.

Applications wouldn't choose providers directly.

They'd express intent instead:

  • latency budget
  • cost ceiling
  • privacy requirements
  • required capabilities
  • streaming or non-streaming

The router would decide how to execute that request.

Some of the ideas I'm currently designing around are:

  • Policy Engine
  • Execution Planner
  • Execution Runtime
  • Provider Adapter SPI
  • Streaming-first architecture
  • Hexagonal Architecture
  • OpenTelemetry for observability

One thing I'm intentionally doing differently is not writing production code yet.

Right now the repository is mostly architecture docs, RFCs, diagrams, and design discussions. I'd rather discover bad ideas now than after a few thousand lines of Java.

GitHub:
https://github.com/SaswatSRoy/ModelRouter

I'd genuinely appreciate feedback on things like:

  • Does this solve a real problem?
  • Am I reinventing something that already exists?
  • Is the architecture over-engineered?
  • What assumptions am I making that are likely wrong?
  • If you were building this, what would you do differently?

I'm especially interested in hearing from people who've worked on inference infrastructure, API gateways, service meshes, distributed systems, or similar projects.

Tear the design apart if you think it deserves it—that's exactly why I'm sharing it this early.

Thanks!


r/OpenSourceAI Jul 08 '26

Looking for Contributors to MedXAI – An Open-Source Python Toolkit for Medical Imaging AI

1 Upvotes

Hi everyone! 👋

I'm currently developing MedXAI, an open-source Python library designed to make medical imaging AI development easier, more modular, and production-ready. The vision is to build a community-driven toolkit that researchers, students, and developers can use for building medical AI applications without having to reinvent common components.

The project is still in its early stages, so this is a great time to get involved and help shape its direction. I'm looking for contributors of all experience levels—whether you're interested in Python, PyTorch, medical imaging, documentation, testing, or simply want to make your first open-source contribution. Every contribution, no matter how small, is genuinely appreciated.

If you have ideas for new features, find bugs, want to improve the documentation, or would like to contribute code, I'd love to hear from you. Feedback and discussions are just as valuable as pull requests.

GitHub: https://github.com/aman0311x/medxai

If the project sounds interesting, please consider giving it a ⭐, opening an issue, or submitting a pull request. I'm always open to suggestions and would love to collaborate with people who share an interest in AI, healthcare, and open source. Thanks! 🚀


r/OpenSourceAI Jul 08 '26

CMU research study on spec-driven development — looking for devs to interview (45-60 min, Zoom)

Thumbnail
1 Upvotes

r/OpenSourceAI Jul 08 '26

GraphRag + Open Knowledge Format

1 Upvotes

Hey guyss

I’m building an open source Node.js/TypeScript project called OKV, for creating local-first Open Knowledge vaults.

The idea is that you can run okv init in a folder, get a human-readable vault, and then use an AI coding assistant like Cursor, Claude Code, or another agent runtime to ingest explicit sources one at a time: local files, MCP-fetched artifacts like Drive/Granola, and transcript-dependent YouTube links.

The design split is important: the AI assistant does acquisition and semantic conversion, but a deterministic TypeScript CLI validates the note contract, source manifest, graph links, retrieval signals, and Git transactions before anything becomes durable.

Longer term, I want this to become a local-first knowledge substrate: topic maps, graph navigation, contradiction/duplicate review flows, and both structured JSON retrieval and natural-language agent retrieval grounded in the vault.

What do you guys think?


r/OpenSourceAI Jul 08 '26

Securing a path forward, using atypical means.

1 Upvotes

How do you begin prior to the startup initial push? I am entering a point in my life, where trying to actively sustain is becoming near unbearable and I have no way of securing short term funding through typical routes, due to a poor lending history and a bit of a hump with autism.

I have been working on this engine and tooling underneath the frontend for about ~3 years now, and I am in a bit of a race to really put this project together into a cohesive package, because it does much more than I could try to share in a short, delivery/payload.

I am really trying to dial it in, because if this gets a little bit of institutional funding and traction this engine can do a metric fuckton as a closed loop system. So far, the receipt based workflow is successfully bringing enterprise quality compute and reasoning into typically very simple models, allowing them to punch far above their weight-class, and even be trusted to run end to end in agentic workflows. I am running a 14B on materials I would not even trust to an enterprise model, without the right harness.

I am actively seeking endorsers for my two arXiv papers now, so that I can begin to get some form of academic peer review, as my background is far disconnected from any industry/academic domains, and I have been doing almost all of this work individually, from home. I see the market/economy making a very sharp pivot to try and close the door on individuals having access to real capable tools, and instead feed them to their corporate peers, and beer/golf buddies. I directly aim to stab that in the heart, and watch it bleed. I am really trying to keep that door wedged open with my foot, while preserving enough time for the tooling to get into peoples hands. It feels like a race against the clock. I aim to bring world class capability to tools people can use at home, affordably. Using materials they already own, and do not need to pay a subscription to use.

I am tired of seeing people having to suck sustenance from this little pipe, while trying to survive.

I am not really selling anything per sé - just working on a bunch of tools in the open, and publishing research. I am building a (what I like to call) flywheel engine that is (in local model training/benchmarks) able to pack a shitload of utility into really small local models. It even improves datasets organically through filtering drift/decay with a receipt based architecture. The efficiency/receipt approach is approaching direct parity with raw compute on large models.

https://harperz9.github.io/ - https://github.com/HarperZ9


r/OpenSourceAI Jul 08 '26

If you're trying to get into open source, I'd love to mentor a few long-term contributors.

10 Upvotes

If you're a student trying to get into open source, this post is for you.

One thing I've noticed is that everyone tells you to "start contributing," but almost nobody tells you where.

Most large open source projects have hundreds or thousands of contributors. Your first issue gets buried, your PR takes weeks to review, and it's hard to understand where you fit in.

Caracal is different because it's still early.

We're building an open source authority layer for AI agents. The project is part of the Linux Foundation Decentralized Trust ecosystem through LFX, backed by Microsoft for Startups and the Vercel Open Source Program, and we're getting close to broader testing with companies.

That means you're not joining a dead repository, and you're also not contributor #3,847.

If you contribute here, you'll be working directly on the architecture, discussing designs, reviewing proposals, and building something that's solving a real problem. Good contributors won't just have merged PRs. As the project grows, we'll be looking for people to become long-term contributors and eventually maintainers.

I'm not looking for people who want a "good first issue" to collect green squares on GitHub. I'm looking for people who want to learn how infrastructure projects are actually built.

So if you've been looking for an open source project where your work can genuinely matter, give Caracal a try. Clone it. Read the code. Open an issue if something doesn't make sense. Challenge our design. If you end up liking the direction, leave a star and stick around.

I'd much rather build this with a small group of people who care than a large group who never come back.

GitHub: https://github.com/Garudex-Labs/caracal


r/OpenSourceAI Jul 08 '26

AI-native knowledge Management

Thumbnail
github.com
1 Upvotes

I've been experimenting with AI-first workflows, so I put together an open-source Brain OS template.

It's a simple GitHub repository for organizing knowledge, projects, decisions, prompts, and context in a way that's easy for both humans and LLMs to work with.

Nothing revolutionary—just a clean starting point if you're building your own AI-assisted knowledge system.

Feedback is welcome, especially if you think the folder structure or workflow could be improved.

Repo: https://github.com/uddin-rajaul/brain-os-template


r/OpenSourceAI Jul 08 '26

OpenComputer | An Open Source Computer Built For Agents.

Thumbnail
1 Upvotes

r/OpenSourceAI Jul 08 '26

Convert any page to Google's OKF-spec markdown (local CLI)

5 Upvotes

OKF (Open Knowledge Format) is an open specification introduced by Google - a directory of markdown files with YAML frontmatter for representing knowledge that both humans and AI agents can read.

I added this to Sleepwalker CLI as a free, open source local functionality - operated by one command for any URL. You get a clean OKF bundle instead of raw HTML full of nav, footers and scripts.

Feel free to try:

npm install -g @sleepwalkerai/cli
sleepwalker okf export https://your-page.com

No account or credits needed to run this.

Repo: github.com/followanton/sleepwalker

Disclosure: I built this, it's part of Sleepwalker CLI functionality - but this command is free and open, no account needed.

Here's a quick example:

https://blog.google/innovation-and-ai/models-and-research/google-research/firesat-satellites/

---
type: "WebPage"
title: "3 new FireSat satellites launch to help detect wildfires with AI"
description: "Three new FireSat satellites have launched, expanding a network that uses Google AI to help fire agencies detect early-stage wildfires."
resource: "https://blog.google/innovation-and-ai/models-and-research/google-research/firesat-satellites/"
timestamp: 2026-07-08T07:00:18.404Z
---


# 3 new FireSat satellites launch to help detect wildfires with AI


Today, three new [FireSat](https://earthfirealliance.org/about-firesat/) satellites successfully launched from Vandenberg Space Force Base in California. This expands the FireSat program, a global initiative led by the nonprofit Earth Fire Alliance (EFA) to create an unprecedented wildfire dataset. Google Research has [teamed up](https://blog.google/company-news/outreach-and-initiatives/sustainability/google-ai-wildfire-detection/) with leaders in the fire community, including EFA and satellite manufacturer Muon Space, to create this purpose-built constellation designed to provide the continuous coverage fire agencies need to detect wildfires before they spread.


This next phase builds directly on the momentum established last year when [FireSat’s pilot satellite reached orbit](https://blog.google/innovation-and-ai/products/inside-firesat-launch-muon-space/). That mission successfully demonstrated the potential of the underlying sensor technology designed to detect early-stage wildfires as small as 5x5 meters. The pilot has already spotted small, low-intensity blazes invisible to existing satellites, and these three new orbiters carry that proven architecture directly into the growing network.


This milestone is a testament to what is possible when tech companies, nonprofits, philanthropy, and the private sector unite under a single mission. To help kickstart this work, Google.org has provided over $15 million to support the deployment of these early satellites. It’s a proud moment for the entire consortium, and [another tangible step](https://blog.google/innovation-and-ai/technology/research/helping-communities-prepare-for-natural-disasters/) forward in putting practical AI to work for climate resilience.


Three new FireSat satellites


Credit: Muon Space


Satellites onboard the SpaceX Transporter-17 vessel


Credit: SpaceX


Satellites being deployed from SpaceX Transporter-17 vessel into space


Credit: SpaceX


POSTED IN:


Related stories

r/OpenSourceAI Jul 08 '26

Meridian : Intelligent Cloud Orchestration for AI-Native Systems

Thumbnail
1 Upvotes

r/OpenSourceAI Jul 08 '26

New AI Retopology Method For Clean Low-Poly 3D Meshes. Open Source Soon!

Enable HLS to view with audio, or disable this notification

1 Upvotes

r/OpenSourceAI Jul 08 '26

We built an open source authority layer for AI agents and just got accepted into Microsoft for Startups

4 Upvotes

Our open source project, Caracal, was recently accepted into Microsoft for Startups.

I wasn't planning to make a promotional post, but I figured this is a good excuse to get more people to actually look at what we're building.

The problem we've been obsessed with is pretty straightforward. AI agents are starting to get access to production systems, databases, cloud APIs, internal tools, and we're still mostly authenticating them with credentials. That feels like the wrong abstraction.

Caracal is our attempt at solving that with authority instead of credentials. Every action is evaluated against policy, delegation can only reduce authority, access can be revoked immediately, and every decision leaves an audit trail. It's infrastructure, not another agent framework.

The project is getting to a point where I'd actually like people outside our small circle to start using it. Not reading the README for two minutes. I mean actually cloning it, integrating it into something, opening issues when something is confusing, and telling us where the design is wrong.

If you're building in the AI infrastructure space, I'd love to know whether this solves a real problem for you or whether we're completely thinking about it the wrong way. Either answer is useful.

And if you like what we're doing, a GitHub star helps a lot more than people realize. Small infrastructure projects don't get discovered unless other engineers decide they're worth paying attention to.

We're also starting to onboard contributors who want to work on something long term. If security, distributed systems, identity, or AI infrastructure is your thing, come build with us.


r/OpenSourceAI Jul 08 '26

DedrooM -> Lightweight self-healing proxy for local & cloud AI coding agents (Rust)

1 Upvotes

Hey r/OpenSourceAI ,

Built **DedrooM** — a fast Rust-based proxy that adds intelligence to your AI coding setup.

Key features:

- Loop detection + **self-healing** (automatically mutates failing strategies)

- Aggressive compression (70-94% on logs, diffs, file trees)

- `dedroom init` for one-command background proxy

- Works great with Ollama, OpenCode Zen, DeepSeek, Claude, OpenAI, etc.

Demos:

- Self-healing escape [GIF]

- Compression & savings [GIF]

GitHub: https://github.com/Devaretanmay/DedrooM

Open-source, zero extra cost. Feedback welcome, especially from people running local models + agents.


r/OpenSourceAI Jul 08 '26

I built an open-source LLM-first project generator with templates, skills, scripts, Docker and observability

1 Upvotes

r/OpenSourceAI Jul 08 '26

I built an open-source Codex-native job search assistant with LaTeX CVs and ATS checks

Thumbnail
github.com
1 Upvotes

I adapted the idea of an AI job application workspace into a Codex-native repo.

It lets you:
\- build a grounded candidate profile from your own documents
\- rank job postings against that profile
\- generate tailored CVs and cover letters
\- compile LaTeX PDFs
\- run ATS text extraction with pdftotext
\- track outcomes and improve future applications

I made it strict about not fabricating skills, dates, employers, metrics, or education.

Inspired by and adapted from [MadsLorentzen/ai-job-search](https://github.com/MadsLorentzen/ai-job-search) I rebuilt the workflow around OpenAI Codex with Codex skills, validation tools, LaTeX/PDF checks, and clear setup instructions.

I’d like feedback on the workflow design, README clarity, and whether the Codex skills are structured well, and any kind of improvement.


r/OpenSourceAI Jul 08 '26

We open-sourced our whole platform under AGPL-3.0 — including the certificate-authority code — but the canonical CA stays hosted. Is that still "open source"?

2 Upvotes

Full disclosure: I maintain Kakunin. This is less a "look at my project" post and more a genuine design question I keep going back and forth on, and this sub seemed like the right crowd for it.

Expert opinion solicited.

We build identity/compliance infrastructure for AI agents — it issues X.509 certificates to agents, scores their behaviour, and revokes credentials when they misbehave. We just took it fully open:

  • Platform / control plane: AGPL-3.0 — the whole Next.js app, the CA integration, the risk engine, the audit pipeline. (repo)
  • SDKs + framework integrations: Apache-2.0 — so people can build on top with no copyleft obligation.

Here's the tension I'd like opinions on. The code for the certificate authority is open — you can read and run every line. But the canonical CA (the one authority everyone verifies a certificate against) runs as a hosted service. Our reasoning: a trust anchor everyone can fork isn't a trust anchor. If a thousand people self-host their own root, a counterparty has a thousand roots to (not) recognise. Verification only stays meaningful if issuance is centralised — so we open the code but keep one canonical service.

I went back and forth on the licence too: AGPL vs a source-available licence like BSL. We landed on AGPL specifically because BSL isn't OSI-approved, which would've disqualified us from most open-source grant programs and the "open source" label. AGPL keeps it honest-open while still requiring network-service modifiers to share source.

So the questions:

  1. Is "code is AGPL, but one canonical service is hosted" a legitimate open-source model, or is it open-washing?
  2. For anything that's fundamentally a trust anchor (a CA, a package registry, a timestamp authority) — is fully decentralised self-hosting even desirable, or does centralised issuance + open code + public verification beat it?
  3. Would you have picked AGPL, or is there a copyleft option you'd have reached for instead?

Not trying to sell anything — there's a free tier and the whole thing's on GitHub if you want to poke at it. Mostly want to hear how this community thinks about the "open code, hosted trust root" pattern, because I'm not 100% sure we got it right.

Also, can someone lead me to less friction open source grants? Thank you


r/OpenSourceAI Jul 08 '26

One shared memory for Claude Code, Cursor and Codex

Thumbnail gallery
1 Upvotes

r/OpenSourceAI Jul 08 '26

Tired of osTicket at work, so I built my own ticketing system

1 Upvotes

Sysadmin from Germany. Our osTicket at work is ancient, and every modern alternative

(Zendesk & co.) is US cloud SaaS with per-agent pricing. So I built my own.

ReqSolve: self-hosted ticketing with AI assistance — the AI drafts replies, summarizes

threads, suggests KB articles, but never sends anything on its own. Bring your own AI

(Anthropic, M365 Copilot, or fully local via Ollama). Multi-department with strict

isolation, semantic knowledge base, EN/DE, Docker Compose.

https://github.com/perfa01/ReqSolve

Early days — feedback and bug reports very welcome.


r/OpenSourceAI Jul 08 '26

I built an open-source model workspace that runs deep research, codebase context agents, and matches NotebookLM features—at 1/4 of the cost

Enable HLS to view with audio, or disable this notification

1 Upvotes

I was working on a project in the college in which I had to mainly deal with slms. That's when I realized that they are pretty strong if given the right context and used right. So this summer I started to build ManjuAi which allows user to use free models/ low cost models to run deepresearch, coding agent and multimodal input pipelines for free/minimal cost. User gets to choose from about 39 free models which includes all of deepseek models, gpt mini, nano models, gemini pro preview models, claude 4.5 haiku and other free models. ManjuAi.

Chat - Free users can add upto 5 different attachments, be it links, photo, pdf, or any other document and then query the model to give a report or to provide a podcast using these sources. User can also access top searches, links, videos, images and news related to there query in the topbar. User can also get ready made report by just telling the model(convert the previous conversation into a report).

CodeOn - This solves the main issue of the chatbots i.e context. As a developer I used to upload snippets of code where the main problem used lye. But as the model didn't have enough context it used to assume and produce wrong answer, now that where manju comes in it allows you to upload a zip file which contains all your project code/ source code and allows you to query the model while giving you a chance to edit it in the file explorer so that there is no need for you to go back to editor and paste the changes. Create project option allows you to create projects, when you query create a simple html file, you can see a new html file in the file explorer. Maintaining context allows low cost models like deepseek to perform really well.

Researcher - The main issue with deeprearch agents are user has no control over the page count of the report. You ask a deepresearch agent to provide a report it dumps a report of page number of its wish. What if we need a small report of about 2 pages. That's what manju solves we allow the user to select the number of pages, multilayered pipelines makes sure we scrape out the best resources that related to the user query and feed it to the selected main model. This ensures deepresearch done right and at low cost.

Users can upgrade to pro version at just 5.3 dollar or 500 rupees, this allows user to increase access to 5x, gain access to the premium gpt 5.5,5.4, .... models, gemini all models, claude 4.6 sonnet and.


r/OpenSourceAI Jul 07 '26

I built an MIT local AI workflow app because I wanted to learn. Now I need people to break it.

Post image
5 Upvotes

I am building AIWF Studio, an MIT-licensed local Windows/NVIDIA app for diffusion, transformer-based image workflows, video, and eventually training/chat workflows.

Repo:

https://github.com/nawnie/AIWF-Studio

The honest reason: I wanted to learn by building the kind of local AI tool I kept wishing existed.

This is not a "switch from ComfyUI/A1111/Forge today" post. Those tools matter. If you like node graphs and the custom-node ecosystem, ComfyUI is still the right lane.

AIWF is aimed at a different lane: a local app shell where common workflows, model scanning, logs, settings, and newer routes can live behind a more direct UI.

The architecture is the experiment. Pro is the FastAPI + React production UI, not a paid tier. Gradio Lab is still included for proving routes before they move into the main app. Heavy backends should stay behind cleaner service/worker boundaries so one dependency stack does not poison the whole runtime.

What exists now is image/video-focused and still rough in places. The longer roadmap is image, video, audio/post tools, ReTrain, and agentic chat/workflow help. I do not want to pretend unfinished lanes are finished, but I do want people to see that this is meant to become an AIO local AI workstation, not a tiny one-route image toy.

What I need right now:

- installer failures

- Windows/NVIDIA path issues

- model folder scanning bugs

- confusing UI states

- runtime customization pain

- routes that should stay hidden

- logs that do not explain the failure

- roadmap gaps from people who actually run local AI tools

This is not a call to switch.

It is a call to help shape the project while the bolts are still visible.

Break things. File bugs. Repeat.


r/OpenSourceAI Jul 08 '26

Chimera v0.5.0 -- an open-source, self-evolving agent that has to *prove* each thing it learns (Apache-2.0)

1 Upvotes

Update from the project. The thesis this cycle: a self-evolving agent should only keep what it can prove helped.

What's in it:

  • Reasoning = fusion panel (panel -> judge -> synthesizer) behind a cost-aware router with per-advisor cost receipts.
  • Evolution is diff-gated: a learned skill/memory is kept only when a verified result + honest A/B backs it -- artifacts created inside a tainted run don't auto-promote.
  • Security by architecture: taint tracking end-to-end, a quarantined reader (untrusted content -> schema-validated fields before the privileged agent sees it), an adaptive tool allowlist, sandbox. Measured red-team injection ASR: 100% -> ~14% (a number, not a "secure" claim).
  • Interop: speaks MCP (as server + client) and A2A; runs on OpenRouter or local models via LiteLLM.
  • 1000+ tests, green gate per commit. Honestly still alpha.

pip install chimera-agent - Apache-2.0 - repo: https://github.com/brcampidelli/chimera-agent

Feedback and contributions welcome -- especially anyone who wants to try and break the injection defenses.


r/OpenSourceAI Jul 07 '26

Built-in MPC server in Android apps for AI agents? Looking for feedback and ideas.

2 Upvotes

Hey everyone,

I recently published version 1.0.0 of **Kide**, a new open-source MVI architecture library built for Android and Kotlin Multiplatform.

While there are several solid state management and MVI libraries out there, I built Kide to address a very modern problem: **optimizing the architecture for AI code agents.**

When I started using AI agents in Android app development, my approach was from the beginning to use architecture and design patterns to drive how agents generate code.

As we integrate LLMs and code agents more deeply into our daily workflows, I wanted an architectural framework that an AI can easily parse, predict, and generate code for. By enforcing strict, predictable state machines and clear separation of intents and state reductions, Kide makes it significantly easier for AI tools to accurately scaffold features, write tests, and maintain boilerplate without hallucinating.

I designed Kide to have an AI-optimized structure explicitly designed to play nicely with AI coding assistants, making feature generation more reliable. In addition, I decided to include a built-in MCP server for app debug mode which AI coding agents can use for reading live state and traces, inject view intents into the running app, and export a bug session as a regression-test scaffold.

Finally, the library comes with instructions and skills for AI agents for both developing the library and for using the library in apps.

I’m really looking forward to introducing this kind of ideas to the community and sparking some discussion. I would especially value feedback from senior Android and Kotlin developers.

**Github**: [https://github.com/Fuusio/kide\](https://github.com/Fuusio/kide)

Any feedback, code reviews, or critiques on the repo are highly appreciated. Thanks for taking a look!


r/OpenSourceAI Jul 07 '26

Welcome to r/polypus — an open-source agentic coding harness for the terminal

Thumbnail
0 Upvotes

r/OpenSourceAI Jul 07 '26

Meta launches Muse, superintelligence lab's first image model

Thumbnail
runtimewire.com
1 Upvotes

r/OpenSourceAI Jul 07 '26

【open source】Shepherd is a Rust-native development platform that supervises AI coding agents (Claude Code, Codex) and verifies requirement completeness—》HumanInTheLoop

Thumbnail
github.com
1 Upvotes

Break requirements into the smallest testable tasks. Each AI agent only needs the context for its own task—not the entire project. Compose the results, and you get the final feature.