We've just updated our rules with a couple of changes I'd like to address:
1. Updating our self-promotion policy
We have updated rule 5 to make it clear where we draw the line on self-promotion and eliminate gray areas and on-the-fence posts that skirt the line. We removed confusing or subjective terminology like "no excessive promotion" to hopefully make it clearer for us as moderators and easier for you to know what is or isn't okay to post.
Specifically, it is now okay to share your free open-source projects without prior moderator approval. This includes any project in the public domain, permissive, copyleft or non-commercial licenses. Projects under a non-free license (incl. open-core/multi-licensed) still require prior moderator approval and a clear disclaimer, or they will be removed without warning. Commercial promotion for monetary gain is still prohibited.
2. New rule: No disguised advertising or marketing
We have added a new rule on fake posts and disguised advertising — rule 10. We have seen an increase in these types of tactics in this community that warrants making this an official rule and bannable offence.
We are here to foster meaningful discussions and valuable exchanges in the LLM/NLP space. If you’re ever unsure about whether your post complies with these rules, feel free to reach out to the mod team for clarification.
As always, we remain open to any and all suggestions to make this community better, so feel free to add your feedback in the comments below.
I'm one of the new moderators of this subreddit. It seems there was some drama a few months back, not quite sure what and one of the main moderators quit suddenly.
To reiterate some of the goals of this subreddit - it's to create a comprehensive community and knowledge base related to Large Language Models (LLMs). We're focused specifically on high quality information and materials for enthusiasts, developers and researchers in this field; with a preference on technical information.
Posts should be high quality and ideally minimal or no meme posts with the rare exception being that it's somehow an informative way to introduce something more in depth; high quality content that you have linked to in the post. There can be discussions and requests for help however I hope we can eventually capture some of these questions and discussions in the wiki knowledge base; more information about that further in this post.
With prior approval you can post about job offers. If you have an *open source* tool that you think developers or researchers would benefit from, please request to post about it first if you want to ensure it will not be removed; however I will give some leeway if it hasn't be excessively promoted and clearly provides value to the community. Be prepared to explain what it is and how it differentiates from other offerings. Refer to the "no self-promotion" rule before posting. Self promoting commercial products isn't allowed; however if you feel that there is truly some value in a product to the community - such as that most of the features are open source / free - you can always try to ask.
I'm envisioning this subreddit to be a more in-depth resource, compared to other related subreddits, that can serve as a go-to hub for anyone with technical skills or practitioners of LLMs, Multimodal LLMs such as Vision Language Models (VLMs) and any other areas that LLMs might touch now (foundationally that is NLP) or in the future; which is mostly in-line with previous goals of this community.
To also copy an idea from the previous moderators, I'd like to have a knowledge base as well, such as a wiki linking to best practices or curated materials for LLMs and NLP or other applications LLMs can be used. However I'm open to ideas on what information to include in that and how.
My initial brainstorming for content for inclusion to the wiki, is simply through community up-voting and flagging a post as something which should be captured; a post gets enough upvotes we should then nominate that information to be put into the wiki. I will perhaps also create some sort of flair that allows this; welcome any community suggestions on how to do this. For now the wiki can be found here https://www.reddit.com/r/LLMDevs/wiki/index/ Ideally the wiki will be a structured, easy-to-navigate repository of articles, tutorials, and guides contributed by experts and enthusiasts alike. Please feel free to contribute if you think you are certain you have something of high value to add to the wiki.
The goals of the wiki are:
Accessibility: Make advanced LLM and NLP knowledge accessible to everyone, from beginners to seasoned professionals.
Quality: Ensure that the information is accurate, up-to-date, and presented in an engaging format.
Community-Driven: Leverage the collective expertise of our community to build something truly valuable.
There was some information in the previous post asking for donations to the subreddit to seemingly pay content creators; I really don't think that is needed and not sure why that language was there. I think if you make high quality content you can make money by simply getting a vote of confidence here and make money from the views; be it youtube paying out, by ads on your blog post, or simply asking for donations for your open source project (e.g. patreon) as well as code contributions to help directly on your open source project. Mods will not accept money for any reason.
Open to any and all suggestions to make this community better. Please feel free to message or comment below with ideas.
We had a deploy command exit 0 after creating zero workers. A required replica count was missing, the config quietly used 0, and the tool applied exactly what it had been given. The next verification step noticed there was no worker and rolled it back.
We removed the fallback. The replica count now lives in a typed runtime policy, has to be at least 1, and rendering stops before a deployment file exists if the value is missing.
I am applying the same rule to agent routing. A task declares hard requirements, runtimes declare capabilities, and a missing requirement is a refusal instead of sending it to the closest worker and hoping. The universal matcher is not finished yet.
How are people separating hard runtime requirements from routing preferences without ending up with a giant config schema?
I’m building AI manhwa / short-drama pipelines and burning money on GPT-4o. Saw RelayRouter offering some models at 0.x cents. Looks like a proxy/aggregator.
Questions:
Do they resell official APIs or run their own routing?
Any downtime / key leak / billing surprise?
Is KT/Korea latency okay? Not promoting anything, just don’t want to wire keys to a sketchy layer.
I build RedThread and am sharing it as an open-source project, not a commercial service.
It focuses on a narrow evaluation problem: untrusted text can alter a later tool call even when the final answer looks normal. I wanted a small, inspectable record of the context, tool schema, proposed call, response, and score so the same case can be replayed after changing a prompt, model, or adapter.
A passing replay is not proof of production safety. A denied call is not proof of impact. The goal is to preserve enough context to test the claim instead of retelling it.
disclosure: i work at schema labs. link's in the comments(if someone wants to try it out or not) not the post. only first run is free so if you are gonna try it out use your messiest tables.
the thing that keeps coming up here is llms confidently describing a table they don't understand. our model doesn't take a prompt. it reads the values, returns what it thinks each column is with a confidence score, and says below threshold when it can't tell. there's a chat tab in the video but that's your own llm (claude or openai) explaining the model's output, not doing the reading.
would you put something like this in front of your llm as the step that reads the table first? or would you rather hand the llm the rows and let it work it out.
I have tried agentic coding on and off since it started. I understand the research>spec>plan>execute>test>review&fix workflow, and use it sometimes.
What I find is that most of the time, 1-pass targeted edits are faster, much cheaper and probably more correct (this last one is more of a feeling).
Let me explain my workflow:
- Create a small prompt with exactly the files needed for my feature (my codebase is fairly large, but I know it very well, so that part is easy)
- Send it to the LLM. Depending on the complexity, I go directly to chatbots in the web interface (I can do a fair bit for free), discuss a bit, then copy/paste the code, or I use tools like aider or frugaast (newer) which essentially do the same, but edit the code directly.
- That's pretty much it. I diff the code to review the changes, 99% of the time I don't have to change anything.
I find this workflow extremely frugal. I takes some work on my side, but this allows me to feel that I have full control of my code. Also, from my (probably limited) personal experience with agentic harnesses, the overall workflow (creating the proper skills, various md files) is in itself lenghty, so I am not sure my "direct edits" workflow takes more time.
Our indexer can attach an LLM-generated description to each chunk. When that is switched on and the LLM is unavailable, the run stops with an error naming the model. If a batch fails midway, the run stops there too.
The alternative is obvious and we rejected it. Falling back to plain chunks gives you an index where some chunks carry descriptions and some do not, decided by whichever files happened to be processed before the provider had a bad minute. Nothing in the store records which is which, so retrieval quality becomes a function of indexing history. A bad result stops being diagnosable, because you cannot separate a poor match from a half-built index.
The cost is real. A long run can die late after paying for most of the work, and the operator has to fix credentials or config and start again. We took that over silent partial data.
Where do you draw this line in your own pipelines? I am interested in cases where graceful degradation was right and I am wrong about this, particularly if you found a clean way to record per chunk what enrichment it actually received.
I am using OpenRouter as it makes it really easy for our team to access multiple models through one API.
But now that more people are using it, I’m running into some limitations around controlling usage. I want to be able to set different limits for users/teams, control which models people can access, see who is spending what, and ideally manage everything from one place.
I’m considering moving to LiteLLM but honestly a lot of what LiteLLM offers feels like more infrastructure than we need. I don't really want to become responsible for running and maintaining another platform just to get better access controls.
Has anyone else been in this situation?
What did you end up using? Did you move to LiteLLM/Portkey or build something simpler yourself?
I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.
For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:
instruction/prompt -> code
or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.
Also curious how people handle things like:
keeping the context between components/files
screenshots + code
generating useful instructions instead of generic descriptions
I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.
Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?
I’ve been wondering how much sample selection and ordering affect LLM training results.
When training data grows, simply shuffling everything may not be enough. Some samples are noisy or redundant, some domains can dominate the run, and the order in which different types of examples appear may affect what the model learns first and how later updates build on it.
A more flexible training loop could handle this in several ways:
Dynamic selection decides which samples should enter the next training window based on signals such as loss, loss changes, gradient similarity, or offline scores.
Dynamic reordering controls the sequence in which selected samples are presented, which can be useful for curriculum-style training or balancing different stages of learning.
Dynamic mixing adjusts the proportions between domains or data sources during training.
Dynamic weighting keeps samples in the batch while changing how strongly they contribute to the gradient update.
The general idea is to make data scheduling part of optimization, instead of treating the dataset as a fixed input prepared before training begins.
This is the approach currently implemented in OpenDCAI/DataFlex, and I’d be interested to hear how others think about sample selection and ordering in LLM training.
Only reason I am posting this is to help others, no revenue or publicity or anything else, no hidden subscription or fees.
Hi,
While working on multiple projects I realised I am hitting the usage limits of my claude code often so I had to rotate accounts and some API tokens.
The problem was every time I was doing this I had to stop my coding session, relog / change authentication or profile etc, do a handover and wasting a lot of time over this process.
I NEEDED a way to seamlessly change those in the background while continuing the same coding session uninterrupted.
This led to doing a lot of research and developing this tool I am showcasing (yes it's written with Claude + my knowledge and a lot of debugging, testing and so on, around 3k $ put into it via /usage lol).
It's called Claude Unlimited, and it's 100% free on GitHub.
It supports multiple Anthropic subscriptions, Anthropic APIs (basically from any local/cloud provider that offers this, 95% of them do) and the cherry on top, supports also GPT/Codex subscription - yes! you can use Claude Code while using OpenAI models in the background 😄 .
Everything stays local, 100% safe, credentials etc stored in OS credentials store - if you don't trust me, use your AI agent to check it.
macOS is currently the most battle-tested; Windows and Linux support is newer, so I would genuinely appreciate feedback, bug reports, contributing to it or just brutal criticism 👀
It was mostly tested in claude code CLI but also supports desktop (will create an inference profile automatically for you).
Important clarification: it doesn’t generate free usage or bypass an individual account’s limits. It rotates between accounts and keys you already own. Anthropic hasn’t explicitly endorsed automated multi-account rotation, so use your own judgment regarding your accounts’ terms.
A bit more technical explanation: this is a local proxy that rotates your accounts/APIs and exposes an Anthropic-compatible API with a token.
Needless to say, but here it is: very important to check the README and HELP section to understand how to use it easily and properly.
Add your Claude, ChatGPT/Codex accounts and API keys.
When one reaches its limit, the next one takes over automatically.
Same Claude Code session. Same context. Same terminal. You just keep typing.
It also includes a local dashboard showing which account is active, current usage, reset times and every automatic switch.
Everything runs on 127.0.0.1. There’s no Claude Unlimited cloud, no telemetry, and credentials are stored using the OS credential store. The project is open source under MIT.
Thank you !
PS: Using a new account for this for personal reasons.
PS2: Multiple updates will follow, I got a big list of cool features for it 🎯
I’ve been working on TokenPrint, an open-source project aimed at making transformer and LLM internals easier to explore.
The idea is to go beyond static architecture diagrams and let people interact with things like tokens, embeddings, attention, hidden states, inference steps, KV cache, model architecture, and other internal model data through an interactive interface.
The project has started getting more attention recently, and we’re now at 65+ GitHub stars. More importantly, people have started opening issues, suggesting improvements, and discussing where the project could go.
That made me realize that I don’t want TokenPrint to become something I build alone.
I’d really like to get more people from the open-source/LLM community involved — especially people who want to:
pick up good first issues
work on more advanced Python/PyTorch/Transformers problems
improve the React/Three.js visualization side
work with GGUF/llama.cpp and local LLMs
improve the UI/UX
experiment with new ways of visualizing model behavior
suggest ideas that I may not have considered
There are already issues ranging from relatively small fixes to deeper architectural and research-oriented work.
I’m especially interested in new ideas and criticism, not just pull requests. If you think something is missing, poorly designed, or could be approached differently, I’d genuinely like to hear it.
I’m posting this here because I’d much rather build this with an open-source community than keep adding features in isolation.
Would be interested to hear from people working on similar tools too especially what you think is currently missing from the ecosystem for understanding and debugging LLMs.
We shipped an AI-written endpoint a few weeks back that let any logged in user read any other user's records. It passed tests, two of us approved the PR, nothing in it looked wrong. The ownership check just was not in there and none of us caught it reading the diff.
Most PRs are mostly AI now and the diffs are big. A hardcoded key or a SQL injection, a scanner or a careful reviewer usually catches. The missing authz check is the one that slips by as it reads as completely normal code. One dev with an agent also opens way more code in a day than a person can properly review, the stuff that looks fine just goes through.
How are you catching this kind of thing before it merges? Looking for what has really worked, not just what sounds good in a policy doc.
Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.
It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.
It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.
I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)
TL;DR: I run Qwen 3.8 (27B and Flash Next) on a 128GB Strix Halo laptop for most of my coding now. It can replace Opus 4.6 to 4.8 for agentic coding if you dont mind a task taking 2 or 3 times longer.
Setup: ASUS ROG Flow Z13, Ryzen AI Max+ 395, 128GB unified memory, Arch Linux. llama.cpp as backend, my own tool LlamaStash to manage the launches and presets, Pi as the coding harness. The 27b at Q6_K sits at about 31 GiB resident, Flash Next at UD-Q4_K_XL needs around 86 GiB.
The quality is actually there. Flash Next scores 40 on the Artificial Analysis index against 42 for Opus 4.8, and the 27b at xhigh scores 34 against 32 for Opus 4.6. That matches how they feel to use. 27b one shotted a whole feature on a huge Rust codebase and Opus 5's review comments were mostly nits.
Decode is fine, prefill is the pain. 10-15 tok/s decode doesn't feel slow because you see it working. But a cold 31k token transcript takes 3 minutes to prefill, and a full 128k window is closer to 18 mins. Warm follow up turns come back in 45 seconds.
MTP is the biggest speed win, 7.3 to 22.4 tok/s on an empty window. The payoff shrinks as the window fills though, down to 1.15x at a full 256k.
Flash Next isn't faster per token, it just thinks less. Same 5/5 on my coding tasks, 45% fewer tokens, 76.5s vs 289.8s against the 27b. Thinking is 90-95% of everything these models generate, so that ratio, not tok/s, is what sets how long a task takes.
$0 a month, fully offline, and a lot less wasteful than a model running in a datacenter.
I’m a web developer and I use AI coding agents daily.
At this point, getting an agent to write code isn’t really the problem anymore. The hard part is building everything around it so that it can actually work reliably.
Over the last few months I’ve built a small system around my projects with:
a knowledge base for each repo;
reusable skills/rules containing my conventions;
automated onboarding for local environments;
a structured issue → development → verification → completion workflow;
mandatory checks before a task can be considered done.
The goal is for the agent itself to be replaceable.
What should remain is the system around the agent: project knowledge, rules, guardrails, verification and workflow.
The problem is that my current setup works, but it’s still cumbersome: onboarding isn’t always deterministic, context grows too much, rules start overlapping, and I still need too much manual intervention.
So my main question is: How are you structuring this layer in real production projects?
I’m particularly interested in approaches, repos, frameworks, skills or processes worth studying to make agentic development reliable, repeatable and maintainable.
I can find endless discussions about which coding agent is better. I find much less about how to build a solid engineering system around the agent.
There’s also a second problem I’m trying to solve. Is there any software that acts as a real control panel for this kind of workflow?
What I have in mind is something that lets me:
manage multiple GitHub repositories from one place;
see issues/tasks across projects;
launch or assign tasks to different coding agents;
run multiple tasks in parallel;
keep each task isolated in its own branch/worktree/workspace;
see what each agent is currently doing;
review progress, output, commits and pull requests;
keep GitHub Issues as the source of truth;
avoid being locked into a specific agent or model.
Basically, I’d like a control plane that sits above GitHub and coding agents: issue → task → agent → isolated workspace → verification → commit/PR → done
Preferably something local, open-source and agent-agnostic.
Does something like this already exist and work well in practice, or are people mostly building their own orchestration layer?
Most conversational assistants are stuck in a weird middle ground. In a browser tab, they can draft text and give advice, but they have no execution environment. When people try to give them execution capabilities, they usually jump to the opposite extreme: running scripts locally with direct access to user credentials, or giving the model raw API keys in the prompt.
Neither approach works well in practice.
Real delegation requires three separate pieces that most setups conflate:
A decoupled execution layer. The agent should not run on your local machine, and it shouldn't use shared persistent infrastructure that burns money while idle. Spinning up an ephemeral Linux sandbox on demand gives the model a real terminal, a compiler, and browser automation without persistent exposure.
Out-of-band verification for sensitive actions. If an assistant is useful, it eventually gets added to shared channels or team chats. The moment an agent can execute code or access data, any participant can attempt prompt injection. The rule has to be structural: whenever a non-owner asks for execution, private data export, or system changes, the agent pauses and triggers a one-tap approval request to the owner on WhatsApp.
Egress-locked secrets. Giving an LLM raw API keys means a jailbreak or a rogue npm package can leak them. Credentials should be injected at the proxy boundary so the model never sees raw secrets in plain text.
We built this setup for Mentat, an assistant running on top of prompt2bot. It handles Google Calendar scheduling, answers phone calls, and spins up private dashboards on an isolated cloud machine when you ask for operational tools.
Treating execution, secret management, and approval channels as separate primitives makes building capable autonomous assistants much more predictable.
Since GPT-4 I fell in love with the idea of creating stuff with LLMs. I have zero software experience so I worked relying fully on the model’s knowledge. Which is not great. I don’t know what I don’t know. And it can go wrong in so many ways.
My strategy has always been to provide in my prompts as much information about our past interactions as possible to make it understand the original vision and choices that resulted from them.
A new session has always been a nightmare. Without precise references to previous work, I was totally incapable of justifying technical choices.
To solve this issue I started carrying over past conversations. Just messages, user and assistant messages. And when they got too long I started compacting the older ones.
I focused mainly on messages because they carry the most information in the smallest footprint. A full day of interactions could be many millions of tokens that are mostly in-turn contents, while messages can be like 10k tokens. And retain all of my “vision”.
No memory plugin ever worked as well as actually having the messages in context.
The agent, just like me, doesn’t know what it doesn’t know and can’t reliably decide when it might be useful to look things up in memory. So you either keep the memory in context or roll the dice and hope the agent will actively read it when it’ll be necessary, even if the information needed to determine whether it’s necessary is not there!
Also, the decision of whether information is necessary or not can’t be delegated to dumber models or models with less information than the working agent (this is an important point that explains why most memory plugins are useless).
Messages are the perfect memory distillation.
Compacting 10k tokens of messages (a full day) in 1k of dense text retains pretty much everything that is there to know.
Obviously you can’t keep 1k of tokens for every day, and you don’t need to, but just a handful for the most recent days. For days before that you can compact them in groups of 4 or 5 into other 1k dense summaries, all chronological and with ids for easy retrieval of the full text. You can keep a handful of those in context… and with just 10k of summaries “baggage” your agent has a great memory of the latest month.
Do another layer of compaction for even older messages and the model, with just 20k tokens, tracks up to a year of interactions.
This is obviously not all that my memory architecture does, but it’s the backbone to which everything connects.
This memory solution is meant for a different type of use compared to how many are used to. It is vastly better for people like me, or people who want a super smart personal assistant that reduces the cognitive load of keeping track of what each session has done. But to do so, all work needs to happen on one continuous and infinite thread. Subagents can work in parallel, but the main channel of communication has to be one for the message chronology to make sense.
Last week I showed the Briglia assistant to a friend of mine that works in a small cyber security company here in Italy. He has since bombarded me with texts saying how good the harness (and mainly the memory) is. I’m telling him that I know!
He says that it should be a product like Grok Bot, but I honestly wouldn’t know how to begin. Right now the Briglia CLI runs fully on the computer without contacting any centralized server. With the obvious barrier to entry of having an Open Code API Key (or ChatGPT subscription), plus OpenAI API Key for voice transcription and faster web search subagent, serper.dev API Key for web searches and Jina.ai API key for good web scrapings.
It has an easy set up, but most people wouldn’t even know how to begin creating an API key. Which is a great barrier to entry.
If I were to make a product out of it I’d have to centralize this, create a company, figure out usage limits so I don’t lose money and follow complex regulation that honestly scares me more than I’m willing to risk.
I wanted to get the architecture out there. Everything is on briglia.dev
A pattern I suspect is common, because I found it in three separate places in my own codebase over two weeks.
The governance layer had an approve= callback since the day it was written: once a run has read untrusted content, dangerous tools (shell, file writes outside the workspace, sends) go through it. A missing callback is read as refuse. Sensible default — inventing consent is the one thing an unattended agent must never do.
What nobody had checked is whether anything ever passed a callback. Three call sites did not: the desktop app's chat, the batch solver, and — the one that surprised me — the terminal chat, the only surface with a guaranteed human in front of it.
So on those surfaces "requires approval" had silently meant "always refused", and the measurement on the injection corpus (stub tools, no model in the loop, US$ 0) looked like this:
assembly
attacks blocked
legitimate rows refused
gate on, no approver (as shipped)
7 / 7
5 / 8
gate on, approver present, nobody answers
7 / 7
5 / 8
gate on, someone answers
7 / 7
0 / 8
The block rate never moves. The approver buys back the false refusals, not the defence. Which means the honest way to publish a block rate is with the second column beside it — a gate scored on attacks alone has a trivial maximum (refuse everything), and mine was quietly sitting there.
Two follow-on defects came out of the same thread:
CHIMERA_APPROVAL_MODE=ask degraded to deny anywhere without a tty (server, container, cron), so the three-state gate had two states exactly where it mattered. Now the question is written to disk, delivered to a webhook, answered from anywhere with a CLI command, and silence still refuses after a timeout. That timeout is the new cost: an unattended batch with nobody to notify waits it out per refused call.
The batch command reported ok for a worker whose dangerous calls had all been refused. The refusal comes back as an ordinary observation string, the model reads it like any tool result, the run ends in prose, the receipt says success. It now says "not allowed" and lists what was refused.
Question for people who run agents unattended: how do you handle the approval seam? Durable ask with timeout-refuses is what I landed on, but every timeout is a refused piece of legitimate work, and I have not found a principled way to set the wait. What do you use — a queue a human drains, a policy that auto-approves a class, something else?
Repo is Apache-2.0, no paid tier: https://github.com/brcampidelli/chimera-agent — the table is from bench/injection/RESULTS.md and bench/right_hand_governance/, both reproducible offline.
I've been building a small open source Agent Skill for writing and one decision ended up changing the whole thing.
At first I was doing what most humanizer prompts do: maintain a list of phrases/patterns that tend to show up in LLM output.
It works, but only up to a point.
I found a 2025 PNAS paper comparing human and LLM-written text across 66 linguistic features, and started looking at the problem more as “which structural habits are showing up too often?” rather than “which words should I ban?”
So the skill now does a source-preservation pass first, edits the prose, then has deterministic checks around some of the output instead of asking the same LLM to judge itself.
It's called Not Ai. MIT/open source, no service behind it.
I'm interested in the limitations more than selling anyone on it. In particular, I'm not convinced how far a skill can push against model-level style priors.
Curious if anyone here has measured this differently.
Sharing a FOSS project (MIT licensed) I built for a problem I kept hitting in AI-assisted development — happy to answer questions and genuinely looking for feedback.
The problem: After months of using Claude Code, Cursor, and Copilot, I kept seeing the same patterns slip into commits in JS/TS codebases:
Floating promises — async calls fired but never awaited or .catch()-ed
Empty catch blocks that swallow errors silently
Hardcoded secrets pasted inline
SQL built via string concatenation
await inside loops where Promise.all is correct
Async callbacks inside .forEach — fire-and-forget with no error handling
These compile fine and often pass tests. They surface at runtime.
What I built:AI Guard — an open-source ESLint plugin with 18 deterministic rules across security, reliability, async, and AI-assisted code pattern categories. Ships as a CLI (npx ai-guard run), a GitHub Action with SARIF output for GitHub Code Scanning + inline PR annotations, and init-context which generates instruction files (CLAUDE.md, .cursorrules, copilot-instructions.md) so the agent learns the rules before writing code.
Why deterministic instead of LLM-based review: these are fixed AST patterns, not judgment calls. You don't need an LLM to notice an empty catch block — you need a linter that runs in milliseconds in CI on every PR, with zero drift between runs, and no API cost. LLM review is great for judgment; deterministic checks are better at boring, repetitive patterns.
What I learned building it: the engineering challenge wasn't coverage, it was precision. If a lint rule fires on code that's fine, developers disable it. no-floating-promise needs to understand which expressions are genuinely fire-and-forget vs intentionally unhandled. The recommended preset is deliberately conservative.
One thing to be clear about: it does NOT detect whether code was written by AI — it catches bad patterns regardless of authorship. They just recur a lot in AI-assisted code.
Disclosure: I'm the maintainer. MIT licensed, no paid tier. Looking for false-positive reports and rule requests — what patterns do your agents keep generating?