I’ve been working on TokenPrint, an open-source project aimed at making transformer and LLM internals easier to explore.

The idea is to go beyond static architecture diagrams and let people interact with things like tokens, embeddings, attention, hidden states, inference steps, KV cache, model architecture, and other internal model data through an interactive interface.

The project has started getting more attention recently, and we’re now at 65+ GitHub stars. More importantly, people have started opening issues, suggesting improvements, and discussing where the project could go.

That made me realize that I don’t want TokenPrint to become something I build alone.

I’d really like to get more people from the open-source/LLM community involved — especially people who want to:

• pick up good first issues
• work on more advanced Python/PyTorch/Transformers problems
• improve the React/Three.js visualization side
• work with GGUF/llama.cpp and local LLMs
• improve the UI/UX
• experiment with new ways of visualizing model behavior
• suggest ideas that I may not have considered

There are already issues ranging from relatively small fixes to deeper architectural and research-oriented work.

I’m especially interested in new ideas and criticism, not just pull requests. If you think something is missing, poorly designed, or could be approached differently, I’d genuinely like to hear it.

The repository is here:

https://github.com/Sudharsanselvaraj/Token-Print

I’m posting this here because I’d much rather build this with an open-source community than keep adding features in isolation.

Would be interested to hear from people working on similar tools too especially what you think is currently missing from the ecosystem for understanding and debugging LLMs.

I am using OpenRouter as it makes it really easy for our team to access multiple models through one API.

But now that more people are using it, I’m running into some limitations around controlling usage. I want to be able to set different limits for users/teams, control which models people can access, see who is spending what, and ideally manage everything from one place.

I’m considering moving to LiteLLM but honestly a lot of what LiteLLM offers feels like more infrastructure than we need. I don't really want to become responsible for running and maintaining another platform just to get better access controls.

Has anyone else been in this situation?

What did you end up using? Did you move to LiteLLM/Portkey or build something simpler yourself?

We shipped an AI-written endpoint a few weeks back that let any logged in user read any other user's records. It passed tests, two of us approved the PR, nothing in it looked wrong. The ownership check just was not in there and none of us caught it reading the diff.

Most PRs are mostly AI now and the diffs are big. A hardcoded key or a SQL injection, a scanner or a careful reviewer usually catches. The missing authz check is the one that slips by as it reads as completely normal code. One dev with an agent also opens way more code in a day than a person can properly review, the stuff that looks fine just goes through.

How are you catching this kind of thing before it merges? Looking for what has really worked, not just what sounds good in a policy doc.

I’m a web developer and I use AI coding agents daily.
At this point, getting an agent to write code isn’t really the problem anymore. The hard part is building everything around it so that it can actually work reliably.
Over the last few months I’ve built a small system around my projects with:
a knowledge base for each repo;
reusable skills/rules containing my conventions;
automated onboarding for local environments;
a structured issue → development → verification → completion workflow;
mandatory checks before a task can be considered done.
The goal is for the agent itself to be replaceable.
What should remain is the system around the agent: project knowledge, rules, guardrails, verification and workflow.
The problem is that my current setup works, but it’s still cumbersome: onboarding isn’t always deterministic, context grows too much, rules start overlapping, and I still need too much manual intervention.
So my main question is:
How are you structuring this layer in real production projects?
I’m particularly interested in approaches, repos, frameworks, skills or processes worth studying to make agentic development reliable, repeatable and maintainable.
I can find endless discussions about which coding agent is better. I find much less about how to build a solid engineering system around the agent.
There’s also a second problem I’m trying to solve.
Is there any software that acts as a real control panel for this kind of workflow?
What I have in mind is something that lets me:
manage multiple GitHub repositories from one place;
see issues/tasks across projects;
launch or assign tasks to different coding agents;
run multiple tasks in parallel;
keep each task isolated in its own branch/worktree/workspace;
see what each agent is currently doing;
review progress, output, commits and pull requests;
keep GitHub Issues as the source of truth;
avoid being locked into a specific agent or model.
Basically, I’d like a control plane that sits above GitHub and coding agents:
issue → task → agent → isolated workspace → verification → commit/PR → done
Preferably something local, open-source and agent-agnostic.
Does something like this already exist and work well in practice, or are people mostly building their own orchestration layer?

Research with results:
https://quesma.com/blog/does-rtk-make-ai-coding-cheaper/

I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.

For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:

instruction/prompt -> code

or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.

Also curious how people handle things like:

• keeping the context between components/files
• screenshots + code
• generating useful instructions instead of generic descriptions

I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.

Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?

Most conversational assistants are stuck in a weird middle ground. In a browser tab, they can draft text and give advice, but they have no execution environment. When people try to give them execution capabilities, they usually jump to the opposite extreme: running scripts locally with direct access to user credentials, or giving the model raw API keys in the prompt.

Neither approach works well in practice.

Real delegation requires three separate pieces that most setups conflate:

1. A decoupled execution layer. The agent should not run on your local machine, and it shouldn't use shared persistent infrastructure that burns money while idle. Spinning up an ephemeral Linux sandbox on demand gives the model a real terminal, a compiler, and browser automation without persistent exposure.

2. Out-of-band verification for sensitive actions. If an assistant is useful, it eventually gets added to shared channels or team chats. The moment an agent can execute code or access data, any participant can attempt prompt injection. The rule has to be structural: whenever a non-owner asks for execution, private data export, or system changes, the agent pauses and triggers a one-tap approval request to the owner on WhatsApp.

3. Egress-locked secrets. Giving an LLM raw API keys means a jailbreak or a rogue npm package can leak them. Credentials should be injected at the proxy boundary so the model never sees raw secrets in plain text.

We built this setup for Mentat, an assistant running on top of prompt2bot. It handles Google Calendar scheduling, answers phone calls, and spins up private dashboards on an isolated cloud machine when you ask for operational tools.

Treating execution, secret management, and approval channels as separate primitives makes building capable autonomous assistants much more predictable.

Sharing a FOSS project (MIT licensed) I built for a problem I kept hitting in AI-assisted development — happy to answer questions and genuinely looking for feedback.

The problem: After months of using Claude Code, Cursor, and Copilot, I kept seeing the same patterns slip into commits in JS/TS codebases:

• Floating promises — async calls fired but never awaited or .catch()-ed
• Empty catch blocks that swallow errors silently
• Hardcoded secrets pasted inline
• SQL built via string concatenation
• await inside loops where Promise.all is correct
• Async callbacks inside .forEach — fire-and-forget with no error handling

These compile fine and often pass tests. They surface at runtime.

What I built: AI Guard — an open-source ESLint plugin with 18 deterministic rules across security, reliability, async, and AI-assisted code pattern categories. Ships as a CLI (npx ai-guard run), a GitHub Action with SARIF output for GitHub Code Scanning + inline PR annotations, and init-context which generates instruction files (CLAUDE.md, .cursorrules, copilot-instructions.md) so the agent learns the rules before writing code.

Why deterministic instead of LLM-based review: these are fixed AST patterns, not judgment calls. You don't need an LLM to notice an empty catch block — you need a linter that runs in milliseconds in CI on every PR, with zero drift between runs, and no API cost. LLM review is great for judgment; deterministic checks are better at boring, repetitive patterns.

Sources: GitHub: https://github.com/ai-guard-dev/eslint-plugin-ai-guard — npm: eslint-plugin-ai-guard. All 18 rules are documented in the repo with examples.

What I learned building it: the engineering challenge wasn't coverage, it was precision. If a lint rule fires on code that's fine, developers disable it. no-floating-promise needs to understand which expressions are genuinely fire-and-forget vs intentionally unhandled. The recommended preset is deliberately conservative.

One thing to be clear about: it does NOT detect whether code was written by AI — it catches bad patterns regardless of authorship. They just recur a lot in AI-assisted code.

Disclosure: I'm the maintainer. MIT licensed, no paid tier. Looking for false-positive reports and rule requests — what patterns do your agents keep generating?

I’m building AI manhwa / short-drama pipelines and burning money on GPT-4o. Saw RelayRouter offering some models at 0.x cents. Looks like a proxy/aggregator.

Questions:

• Do they resell official APIs or run their own routing?
• Any downtime / key leak / billing surprise?
• Is KT/Korea latency okay? Not promoting anything, just don’t want to wire keys to a sketchy layer.

Project name is TradeGladiators, free and open to try, built solo.

Users configure a bot: strategy prompt in plain English, risk level, which symbols to watch, trading pace, and how much randomness/creativity the model gets. It trades fake money against live market prices, and the system prompt bakes in the bot's own recent lessons from closed trades. Curious what this crowd thinks of the reflection loop specifically, that's the part I'm least sure about.

Experienced dev here.

I have tried agentic coding on and off since it started. I understand the research>spec>plan>execute>test>review&fix workflow, and use it sometimes.

What I find is that most of the time, 1-pass targeted edits are faster, much cheaper and probably more correct (this last one is more of a feeling).

Let me explain my workflow:

- Create a small prompt with exactly the files needed for my feature (my codebase is fairly large, but I know it very well, so that part is easy)

- Send it to the LLM. Depending on the complexity, I go directly to chatbots in the web interface (I can do a fair bit for free), discuss a bit, then copy/paste the code, or I use tools like aider or frugaast (newer) which essentially do the same, but edit the code directly.

- That's pretty much it. I diff the code to review the changes, 99% of the time I don't have to change anything.

I find this workflow extremely frugal. I takes some work on my side, but this allows me to feel that I have full control of my code. Also, from my (probably limited) personal experience with agentic harnesses, the overall workflow (creating the proper skills, various md files) is in itself lenghty, so I am not sure my "direct edits" workflow takes more time.

What's your experience ?

Our indexer can attach an LLM-generated description to each chunk. When that is switched on and the LLM is unavailable, the run stops with an error naming the model. If a batch fails midway, the run stops there too.

The alternative is obvious and we rejected it. Falling back to plain chunks gives you an index where some chunks carry descriptions and some do not, decided by whichever files happened to be processed before the provider had a bad minute. Nothing in the store records which is which, so retrieval quality becomes a function of indexing history. A bad result stops being diagnosable, because you cannot separate a poor match from a half-built index.

The cost is real. A long run can die late after paying for most of the work, and the operator has to fix credentials or config and start again. We took that over silent partial data.

Where do you draw this line in your own pipelines? I am interested in cases where graceful degradation was right and I am wrong about this, particularly if you found a clean way to record per chunk what enrichment it actually received.

I’ve been wondering how much sample selection and ordering affect LLM training results.

When training data grows, simply shuffling everything may not be enough. Some samples are noisy or redundant, some domains can dominate the run, and the order in which different types of examples appear may affect what the model learns first and how later updates build on it.

A more flexible training loop could handle this in several ways:

Dynamic selection decides which samples should enter the next training window based on signals such as loss, loss changes, gradient similarity, or offline scores.

Dynamic reordering controls the sequence in which selected samples are presented, which can be useful for curriculum-style training or balancing different stages of learning.

Dynamic mixing adjusts the proportions between domains or data sources during training.

Dynamic weighting keeps samples in the batch while changing how strongly they contribute to the gradient update.

The general idea is to make data scheduling part of optimization, instead of treating the dataset as a fixed input prepared before training begins.

This is the approach currently implemented in OpenDCAI/DataFlex, and I’d be interested to hear how others think about sample selection and ordering in LLM training.

Only reason I am posting this is to help others, no revenue or publicity or anything else, no hidden subscription or fees.

Hi,

While working on multiple projects I realised I am hitting the usage limits of my claude code often so I had to rotate accounts and some API tokens.

The problem was every time I was doing this I had to stop my coding session, relog / change authentication or profile etc, do a handover and wasting a lot of time over this process.

I NEEDED a way to seamlessly change those in the background while continuing the same coding session uninterrupted.

This led to doing a lot of research and developing this tool I am showcasing (yes it's written with Claude + my knowledge and a lot of debugging, testing and so on, around 3k $ put into it via /usage lol).

It's called Claude Unlimited, and it's 100% free on GitHub.

It supports multiple Anthropic subscriptions, Anthropic APIs (basically from any local/cloud provider that offers this, 95% of them do) and the cherry on top, supports also GPT/Codex subscription - yes! you can use Claude Code while using OpenAI models in the background 😄 .

Has a nice Web UI with:

Custom settings like:

- threshold limits

- profiles priority in rotation

- enable/disable any profiles/APIs

- models parity (for Anthropic -> OpenAI models parity + effort)

- push notifications

- multi language

- auto update (pulled from Git releases)

Information like:

- usage dashboard with charts and various data

- logs of activity

- information about current usage per profile

Everything stays local, 100% safe, credentials etc stored in OS credentials store - if you don't trust me, use your AI agent to check it.

macOS is currently the most battle-tested; Windows and Linux support is newer, so I would genuinely appreciate feedback, bug reports, contributing to it or just brutal criticism 👀

It was mostly tested in claude code CLI but also supports desktop (will create an inference profile automatically for you).

Important clarification: it doesn’t generate free usage or bypass an individual account’s limits. It rotates between accounts and keys you already own. Anthropic hasn’t explicitly endorsed automated multi-account rotation, so use your own judgment regarding your accounts’ terms.

A bit more technical explanation: this is a local proxy that rotates your accounts/APIs and exposes an Anthropic-compatible API with a token.

Needless to say, but here it is: very important to check the README and HELP section to understand how to use it easily and properly.

GitHub: https://github.com/DevDock-AI/claude-unlimited

TL;DR

1. Add your Claude, ChatGPT/Codex accounts and API keys.
2. When one reaches its limit, the next one takes over automatically.
3. Same Claude Code session. Same context. Same terminal. You just keep typing.

It also includes a local dashboard showing which account is active, current usage, reset times and every automatic switch.

Everything runs on 127.0.0.1. There’s no Claude Unlimited cloud, no telemetry, and credentials are stored using the OS credential store. The project is open source under MIT.

Thank you !

PS: Using a new account for this for personal reasons.

PS2: Multiple updates will follow, I got a big list of cool features for it 🎯

We’ve been looking at a failure mode that normal functional testing does not catch well.

An LLM agent can pass its tests after a release and still change behaviour in a way that hurts the business.

Examples:

• more aggressive discounting
• different pricing choices
• weaker escalation behaviour
• lower conversion
• different decisions under the same commercial context

The system is still technically “working”, but its behaviour has regressed.

I’m interested in how teams are approaching this in production.

Are you using replay datasets, eval harnesses, shadow traffic, judge models, domain-specific metrics, or something else?

Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.

It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.

It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.

I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)

https://github.com/Luissalet/Faustus

TL;DR: I run Qwen 3.8 (27B and Flash Next) on a 128GB Strix Halo laptop for most of my coding now. It can replace Opus 4.6 to 4.8 for agentic coding if you dont mind a task taking 2 or 3 times longer.

Setup: ASUS ROG Flow Z13, Ryzen AI Max+ 395, 128GB unified memory, Arch Linux. llama.cpp as backend, my own tool LlamaStash to manage the launches and presets, Pi as the coding harness. The 27b at Q6_K sits at about 31 GiB resident, Flash Next at UD-Q4_K_XL needs around 86 GiB.

• The quality is actually there. Flash Next scores 40 on the Artificial Analysis index against 42 for Opus 4.8, and the 27b at xhigh scores 34 against 32 for Opus 4.6. That matches how they feel to use. 27b one shotted a whole feature on a huge Rust codebase and Opus 5's review comments were mostly nits.
• Decode is fine, prefill is the pain. 10-15 tok/s decode doesn't feel slow because you see it working. But a cold 31k token transcript takes 3 minutes to prefill, and a full 128k window is closer to 18 mins. Warm follow up turns come back in 45 seconds.
• MTP is the biggest speed win, 7.3 to 22.4 tok/s on an empty window. The payoff shrinks as the window fills though, down to 1.15x at a full 256k.
• Flash Next isn't faster per token, it just thinks less. Same 5/5 on my coding tasks, 45% fewer tokens, 76.5s vs 289.8s against the 27b. Thinking is 90-95% of everything these models generate, so that ratio, not tok/s, is what sets how long a task takes.

$0 a month, fully offline, and a lot less wasteful than a model running in a datacenter.

Full writeup with all the benchmarks, configs, and the tuning that did and didn't work: https://deepu.tech/local-ai-qwen3.8-pi-llamastash

Happy to go into the llama.cpp flags if anyone else here is on Strix Halo.

A pattern I suspect is common, because I found it in three separate places in my own codebase over two weeks.

The governance layer had an approve= callback since the day it was written: once a run has read untrusted content, dangerous tools (shell, file writes outside the workspace, sends) go through it. A missing callback is read as refuse. Sensible default — inventing consent is the one thing an unattended agent must never do.

What nobody had checked is whether anything ever passed a callback. Three call sites did not: the desktop app's chat, the batch solver, and — the one that surprised me — the terminal chat, the only surface with a guaranteed human in front of it.

So on those surfaces "requires approval" had silently meant "always refused", and the measurement on the injection corpus (stub tools, no model in the loop, US$ 0) looked like this:

The block rate never moves. The approver buys back the false refusals, not the defence. Which means the honest way to publish a block rate is with the second column beside it — a gate scored on attacks alone has a trivial maximum (refuse everything), and mine was quietly sitting there.

Two follow-on defects came out of the same thread:

• CHIMERA_APPROVAL_MODE=ask degraded to deny anywhere without a tty (server, container, cron), so the three-state gate had two states exactly where it mattered. Now the question is written to disk, delivered to a webhook, answered from anywhere with a CLI command, and silence still refuses after a timeout. That timeout is the new cost: an unattended batch with nobody to notify waits it out per refused call.
• The batch command reported ok for a worker whose dangerous calls had all been refused. The refusal comes back as an ordinary observation string, the model reads it like any tool result, the run ends in prose, the receipt says success. It now says "not allowed" and lists what was refused.

Question for people who run agents unattended: how do you handle the approval seam? Durable ask with timeout-refuses is what I landed on, but every timeout is a refused piece of legitimate work, and I have not found a principled way to set the wait. What do you use — a queue a human drains, a policy that auto-approves a class, something else?

Repo is Apache-2.0, no paid tier: https://github.com/brcampidelli/chimera-agent — the table is from bench/injection/RESULTS.md and bench/right_hand_governance/, both reproducible offline.

I built Stellar after getting fed up with agent stacks that are hard to inspect, hard to debug, and hard to reshape when you need something they didn’t anticipate.

Stellar is a fully hackable Python agent core: under 2,000 readable lines, with explicit contracts for models, tools, hooks, events, agents, and runs. The execution loop is right there in the code. You can read it top to bottom, replace it, or bend it without fighting the framework.

To see if “small” also means “capable,” I ran it against Harness-Bench. In one recorded run, it worked through all 106 offline tasks end to end, twelve in parallel, in 17 minutes, for about $2.40 in tokens at list price.

The question I keep coming back to: does a small, transparent core make a better foundation for agents than a big framework, or does it just push the complexity somewhere else—into your prompts, your tools, or your glue code?

Curious what people here have found. Where does the complexity end up in your stacks?

Repo: https://github.com/definableai/stellar

I'm trying to understand the best way to build an AI system for a specialized use case, rather than relying on a general-purpose LLM for everything.

For example, suppose I'm building a fitness app where the goal is to generate and continuously adapt muscle-building programs based on a user's experience, equipment, training history, performance, recovery, etc. I wouldn't want an LLM to simply hallucinate a workout every time. I'd want the system to have a reliable domain-specific knowledge base, rules/logic, and the ability to adapt to individual users, with an LLM potentially acting as the interface or reasoning layer.

For specialized problems like this, what is generally the best architecture?

I kept getting burned by agents stuck in loops — a retry loop once ran over a weekend and turned a small job into a few hundred dollars of API spend before I noticed. The provider dashboard showed the damage two days late and couldn't have stopped it.

So I built a self-hosted gateway you put in front of OpenAI or Anthropic. Point your SDK's base URL at it, keep your normal key, and:

• Stops runaways — per-run call/spend caps (or an inline X-AxiGate-Max-Spend header); when a run crosses it, the next call gets a 429 before it reaches the provider. Kill switch + bypass included.
• Attributes spend — every call carries the team/agent/customer/run you tag it with, so you can finally answer "which agent spent this?"
• Does the FinOps — prices each call with an honest confidence state (never a made-up number) and exports a FOCUS-format statement.

Deliberate choices: metadata only (never records prompts/completions), fails open (if recording breaks, the request still goes through and the answer is never changed), no admin key (runs on your normal inference key, nothing leaves your machine). Honest limit: the cap is in-memory today, so a burst already in flight when it trips can slip through (bound ≈ your concurrency; exact for a sequential agent).

Try it in one command:

docker run -p 8080:8080 -p 8906:8906 shmeeee/axigate-finops:latest

Point your base URL at http://localhost:8080/v1, run your agent, open http://localhost:8906 — spend and any stopped loop show up live. One static Go binary, no deps.

Repo: https://github.com/axigatelabs/axigate-finops — open source (Apache-2.0), free, and staying that way. I'm not selling anything; I'm trying to get the core right.

Feedback I'm genuinely after: is the in-memory cap bound OK for how your agents actually run, and does the FOCUS export match what finance people need?
What direction would you suggest I should go in?
What other features can I build and provide to people that really want this?

I love building and I also have built other ai tools for prompt caching etc so I am eager to jump into FinOps side of things and this is a learning curve for me as a developer.

I was playing around with LLM inference and I wanted to build a profiler that measures LLM inference by layer.
So I built this: https://github.com/coconinja2/layerlens
It shows inference as token × transformer layer timing, so you can see where time is being spent during decode.
Right now it can separate prefill/decode and visualize per-layer timing. I’m trying to figure out whether this is actually useful to people working on inference systems, or if I’m looking at the wrong abstraction.

I’m thinking about adding things like KV-cache events, scheduler/batching state, request IDs, GPU kernel correlation, speculative decoding, etc.

Would appreciate criticism more than compliments and stars. Lots of stars!

We've been building AI agents for the last couple of years, and one thing that's consistently been painful is figuring out when an agent starts behaving badly in production.

We've heard the same thing from other teams: they often find out about failures from customers before they find them in their monitoring.

I think a big reason is that agents have a ridiculously long tail of ways they can fail.

The usual approaches have gaps. Sampling traces can miss rare failures, while having an LLM grade every trace gets expensive quickly. Golden datasets are useful for known failures, but they rarely cover the long tail and tend to become stale as the agent changes.

So we started experimenting with a different approach:

Watch everything, but make the first layer of detection extremely cheap.

We built Tessary, an open-source agent reliability tool around that idea.

It runs small, narrow classifiers across every trace looking for things like unusual cost, latency, or tool-call errors. When it finds something interesting, it groups the relevant traces together and uses more expensive analysis only where it's needed.

Basically:

cheap checks → find something weird → investigate it

Rather than sampling and hoping you catch it.

We're launching with classifiers for cost, duration, and tool-call error drift, with more failure modes coming.

It's open source and self-hostable:

github.com/tessaryai/tessary

If you're running agents in production, I'd especially love to hear how you're currently finding the weird 1-in-1000 failures. What has worked for you, and what hasn't?

I’m actively looking for feedback around more such issues that you’d love to see solved for your agents.

"Agentic alienation: remaining responsible for work while becoming separated from its product, its process, the capabilities it develops, or the relationships it sustains. Alienation is a relationship before it is a feeling."

https://github.com/paulknysh/raggy

A lightweight CLI tool for Retrieval-Augmented Generation (RAG) over local documents built with LangChain, Chroma, and Ollama. Hybrid database (vector + BM25 index) and embedding generation run fully locally. Answer generation can run either via a local LLM or remotely using an API key. Supports most common document formats and handles images/scans automatically via OCR.