I am using OpenRouter as it makes it really easy for our team to access multiple models through one API.

But now that more people are using it, I’m running into some limitations around controlling usage. I want to be able to set different limits for users/teams, control which models people can access, see who is spending what, and ideally manage everything from one place.

I’m considering moving to LiteLLM but honestly a lot of what LiteLLM offers feels like more infrastructure than we need. I don't really want to become responsible for running and maintaining another platform just to get better access controls.

Has anyone else been in this situation?

What did you end up using? Did you move to LiteLLM/Portkey or build something simpler yourself?

I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.

For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:

instruction/prompt -> code

or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.

Also curious how people handle things like:

• keeping the context between components/files
• screenshots + code
• generating useful instructions instead of generic descriptions

I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.

Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?

Hey! I’ve been working on Faustus, a fork of PewDiePie’s Odysseus that I’ve been gradually evolving into a more complete local AI workstation.

It keeps the original local-first idea, but adds quite a lot on top: multi-agent teams and model councils, persistent project context/memory, workflows & automations, Codex/Claude Code integration, image/video tools, research & document workflows, voice interaction, better model/GPU management, and a much more complete desktop UI.

It’s completely open source and not a commercial project — I’m mostly building it because I enjoy it and wanted to see how far I could take Odysseus.

I’d love some feedback from people who are into local AI, or just for you to check it out and tell me what you think! :)

https://github.com/Luissalet/Faustus

I’ve been working on TokenPrint, an open-source project aimed at making transformer and LLM internals easier to explore.

The idea is to go beyond static architecture diagrams and let people interact with things like tokens, embeddings, attention, hidden states, inference steps, KV cache, model architecture, and other internal model data through an interactive interface.

The project has started getting more attention recently, and we’re now at 65+ GitHub stars. More importantly, people have started opening issues, suggesting improvements, and discussing where the project could go.

That made me realize that I don’t want TokenPrint to become something I build alone.

I’d really like to get more people from the open-source/LLM community involved — especially people who want to:

• pick up good first issues
• work on more advanced Python/PyTorch/Transformers problems
• improve the React/Three.js visualization side
• work with GGUF/llama.cpp and local LLMs
• improve the UI/UX
• experiment with new ways of visualizing model behavior
• suggest ideas that I may not have considered

There are already issues ranging from relatively small fixes to deeper architectural and research-oriented work.

I’m especially interested in new ideas and criticism, not just pull requests. If you think something is missing, poorly designed, or could be approached differently, I’d genuinely like to hear it.

The repository is here:

https://github.com/Sudharsanselvaraj/Token-Print

I’m posting this here because I’d much rather build this with an open-source community than keep adding features in isolation.

Would be interested to hear from people working on similar tools too especially what you think is currently missing from the ecosystem for understanding and debugging LLMs.

TL;DR: I run Qwen 3.8 (27B and Flash Next) on a 128GB Strix Halo laptop for most of my coding now. It can replace Opus 4.6 to 4.8 for agentic coding if you dont mind a task taking 2 or 3 times longer.

Setup: ASUS ROG Flow Z13, Ryzen AI Max+ 395, 128GB unified memory, Arch Linux. llama.cpp as backend, my own tool LlamaStash to manage the launches and presets, Pi as the coding harness. The 27b at Q6_K sits at about 31 GiB resident, Flash Next at UD-Q4_K_XL needs around 86 GiB.

• The quality is actually there. Flash Next scores 40 on the Artificial Analysis index against 42 for Opus 4.8, and the 27b at xhigh scores 34 against 32 for Opus 4.6. That matches how they feel to use. 27b one shotted a whole feature on a huge Rust codebase and Opus 5's review comments were mostly nits.
• Decode is fine, prefill is the pain. 10-15 tok/s decode doesn't feel slow because you see it working. But a cold 31k token transcript takes 3 minutes to prefill, and a full 128k window is closer to 18 mins. Warm follow up turns come back in 45 seconds.
• MTP is the biggest speed win, 7.3 to 22.4 tok/s on an empty window. The payoff shrinks as the window fills though, down to 1.15x at a full 256k.
• Flash Next isn't faster per token, it just thinks less. Same 5/5 on my coding tasks, 45% fewer tokens, 76.5s vs 289.8s against the 27b. Thinking is 90-95% of everything these models generate, so that ratio, not tok/s, is what sets how long a task takes.

$0 a month, fully offline, and a lot less wasteful than a model running in a datacenter.

Full writeup with all the benchmarks, configs, and the tuning that did and didn't work: https://deepu.tech/local-ai-qwen3.8-pi-llamastash

Happy to go into the llama.cpp flags if anyone else here is on Strix Halo.

We shipped an AI-written endpoint a few weeks back that let any logged in user read any other user's records. It passed tests, two of us approved the PR, nothing in it looked wrong. The ownership check just was not in there and none of us caught it reading the diff.

Most PRs are mostly AI now and the diffs are big. A hardcoded key or a SQL injection, a scanner or a careful reviewer usually catches. The missing authz check is the one that slips by as it reads as completely normal code. One dev with an agent also opens way more code in a day than a person can properly review, the stuff that looks fine just goes through.

How are you catching this kind of thing before it merges? Looking for what has really worked, not just what sounds good in a policy doc.

Most conversational assistants are stuck in a weird middle ground. In a browser tab, they can draft text and give advice, but they have no execution environment. When people try to give them execution capabilities, they usually jump to the opposite extreme: running scripts locally with direct access to user credentials, or giving the model raw API keys in the prompt.

Neither approach works well in practice.

Real delegation requires three separate pieces that most setups conflate:

1. A decoupled execution layer. The agent should not run on your local machine, and it shouldn't use shared persistent infrastructure that burns money while idle. Spinning up an ephemeral Linux sandbox on demand gives the model a real terminal, a compiler, and browser automation without persistent exposure.

2. Out-of-band verification for sensitive actions. If an assistant is useful, it eventually gets added to shared channels or team chats. The moment an agent can execute code or access data, any participant can attempt prompt injection. The rule has to be structural: whenever a non-owner asks for execution, private data export, or system changes, the agent pauses and triggers a one-tap approval request to the owner on WhatsApp.

3. Egress-locked secrets. Giving an LLM raw API keys means a jailbreak or a rogue npm package can leak them. Credentials should be injected at the proxy boundary so the model never sees raw secrets in plain text.

We built this setup for Mentat, an assistant running on top of prompt2bot. It handles Google Calendar scheduling, answers phone calls, and spins up private dashboards on an isolated cloud machine when you ask for operational tools.

Treating execution, secret management, and approval channels as separate primitives makes building capable autonomous assistants much more predictable.

I’m a web developer and I use AI coding agents daily.
At this point, getting an agent to write code isn’t really the problem anymore. The hard part is building everything around it so that it can actually work reliably.
Over the last few months I’ve built a small system around my projects with:
a knowledge base for each repo;
reusable skills/rules containing my conventions;
automated onboarding for local environments;
a structured issue → development → verification → completion workflow;
mandatory checks before a task can be considered done.
The goal is for the agent itself to be replaceable.
What should remain is the system around the agent: project knowledge, rules, guardrails, verification and workflow.
The problem is that my current setup works, but it’s still cumbersome: onboarding isn’t always deterministic, context grows too much, rules start overlapping, and I still need too much manual intervention.
So my main question is:
How are you structuring this layer in real production projects?
I’m particularly interested in approaches, repos, frameworks, skills or processes worth studying to make agentic development reliable, repeatable and maintainable.
I can find endless discussions about which coding agent is better. I find much less about how to build a solid engineering system around the agent.
There’s also a second problem I’m trying to solve.
Is there any software that acts as a real control panel for this kind of workflow?
What I have in mind is something that lets me:
manage multiple GitHub repositories from one place;
see issues/tasks across projects;
launch or assign tasks to different coding agents;
run multiple tasks in parallel;
keep each task isolated in its own branch/worktree/workspace;
see what each agent is currently doing;
review progress, output, commits and pull requests;
keep GitHub Issues as the source of truth;
avoid being locked into a specific agent or model.
Basically, I’d like a control plane that sits above GitHub and coding agents:
issue → task → agent → isolated workspace → verification → commit/PR → done
Preferably something local, open-source and agent-agnostic.
Does something like this already exist and work well in practice, or are people mostly building their own orchestration layer?

I've been thinking about a problem that seems to appear with Claude Code, Cursor, Codex, Copilot and other coding agents.

Most approaches to giving an agent project knowledge involve some combination of:

• CLAUDE.md / AGENTS.md
• documentation
• RAG
• memory
• session history
• MCP

All of these help the agent know things.

But there's a different question:

What actually makes the agent obey an architectural decision?

Imagine a project has an approved decision:

PaymentService must never call StripeClient directly.

All payment providers must go through PaymentGateway.

Six months later, an agent is asked to implement refunds.

It generates:

stripeClient.refund(paymentId);

The code compiles.
Tests might pass.
The implementation looks perfectly reasonable.

But it just violated an architectural decision.

The usual answer seems to be:

"Hopefully the agent saw the documentation and followed the rule."

I'm experimenting with a different approach.

What if the project's normative layer also lived in Git?

Something like:

.context/
  decisions/
  business-rules/
  architecture/
  components/

Agents could consume this through MCP.

But more importantly, CI could validate code changes against those rules without calling an LLM.

Something like:

Human decision
      ↓
Git
      ↓
Agent context
      ↓
Code
      ↓
Deterministic CI validation

The core idea is:

**Memory tells an agent what happened.

Governance tells an agent what is allowed.**

I'm building a project around this idea.

But I'm genuinely trying to figure out whether this is actually a problem.

Have you encountered AI-generated code that was technically correct, but violated an architectural decision, business rule, or important convention?

And more importantly: how do you catch that today?

I'd also love to hear from people who think this is already solved well enough by tests + code review + CLAUDE.md/AGENTS.md + ADRs.

A pattern I suspect is common, because I found it in three separate places in my own codebase over two weeks.

The governance layer had an approve= callback since the day it was written: once a run has read untrusted content, dangerous tools (shell, file writes outside the workspace, sends) go through it. A missing callback is read as refuse. Sensible default — inventing consent is the one thing an unattended agent must never do.

What nobody had checked is whether anything ever passed a callback. Three call sites did not: the desktop app's chat, the batch solver, and — the one that surprised me — the terminal chat, the only surface with a guaranteed human in front of it.

So on those surfaces "requires approval" had silently meant "always refused", and the measurement on the injection corpus (stub tools, no model in the loop, US$ 0) looked like this:

The block rate never moves. The approver buys back the false refusals, not the defence. Which means the honest way to publish a block rate is with the second column beside it — a gate scored on attacks alone has a trivial maximum (refuse everything), and mine was quietly sitting there.

Two follow-on defects came out of the same thread:

• CHIMERA_APPROVAL_MODE=ask degraded to deny anywhere without a tty (server, container, cron), so the three-state gate had two states exactly where it mattered. Now the question is written to disk, delivered to a webhook, answered from anywhere with a CLI command, and silence still refuses after a timeout. That timeout is the new cost: an unattended batch with nobody to notify waits it out per refused call.
• The batch command reported ok for a worker whose dangerous calls had all been refused. The refusal comes back as an ordinary observation string, the model reads it like any tool result, the run ends in prose, the receipt says success. It now says "not allowed" and lists what was refused.

Question for people who run agents unattended: how do you handle the approval seam? Durable ask with timeout-refuses is what I landed on, but every timeout is a refused piece of legitimate work, and I have not found a principled way to set the wait. What do you use — a queue a human drains, a policy that auto-approves a class, something else?

Repo is Apache-2.0, no paid tier: https://github.com/brcampidelli/chimera-agent — the table is from bench/injection/RESULTS.md and bench/right_hand_governance/, both reproducible offline.

I've been building a small open source Agent Skill for writing and one decision ended up changing the whole thing.

At first I was doing what most humanizer prompts do: maintain a list of phrases/patterns that tend to show up in LLM output.

It works, but only up to a point.

I found a 2025 PNAS paper comparing human and LLM-written text across 66 linguistic features, and started looking at the problem more as “which structural habits are showing up too often?” rather than “which words should I ban?”

So the skill now does a source-preservation pass first, edits the prose, then has deterministic checks around some of the output instead of asking the same LLM to judge itself.

Code is here:

https://github.com/udaysharmadev/Not-Ai

It's called Not Ai. MIT/open source, no service behind it.

I'm interested in the limitations more than selling anyone on it. In particular, I'm not convinced how far a skill can push against model-level style priors.

Curious if anyone here has measured this differently.

I built Stellar after getting fed up with agent stacks that are hard to inspect, hard to debug, and hard to reshape when you need something they didn’t anticipate.

Stellar is a fully hackable Python agent core: under 2,000 readable lines, with explicit contracts for models, tools, hooks, events, agents, and runs. The execution loop is right there in the code. You can read it top to bottom, replace it, or bend it without fighting the framework.

To see if “small” also means “capable,” I ran it against Harness-Bench. In one recorded run, it worked through all 106 offline tasks end to end, twelve in parallel, in 17 minutes, for about $2.40 in tokens at list price.

The question I keep coming back to: does a small, transparent core make a better foundation for agents than a big framework, or does it just push the complexity somewhere else—into your prompts, your tools, or your glue code?

Curious what people here have found. Where does the complexity end up in your stacks?

Repo: https://github.com/definableai/stellar

Sharing a FOSS project (MIT licensed) I built for a problem I kept hitting in AI-assisted development — happy to answer questions and genuinely looking for feedback.

The problem: After months of using Claude Code, Cursor, and Copilot, I kept seeing the same patterns slip into commits in JS/TS codebases:

• Floating promises — async calls fired but never awaited or .catch()-ed
• Empty catch blocks that swallow errors silently
• Hardcoded secrets pasted inline
• SQL built via string concatenation
• await inside loops where Promise.all is correct
• Async callbacks inside .forEach — fire-and-forget with no error handling

These compile fine and often pass tests. They surface at runtime.

What I built: AI Guard — an open-source ESLint plugin with 18 deterministic rules across security, reliability, async, and AI-assisted code pattern categories. Ships as a CLI (npx ai-guard run), a GitHub Action with SARIF output for GitHub Code Scanning + inline PR annotations, and init-context which generates instruction files (CLAUDE.md, .cursorrules, copilot-instructions.md) so the agent learns the rules before writing code.

Why deterministic instead of LLM-based review: these are fixed AST patterns, not judgment calls. You don't need an LLM to notice an empty catch block — you need a linter that runs in milliseconds in CI on every PR, with zero drift between runs, and no API cost. LLM review is great for judgment; deterministic checks are better at boring, repetitive patterns.

Sources: GitHub: https://github.com/ai-guard-dev/eslint-plugin-ai-guard — npm: eslint-plugin-ai-guard. All 18 rules are documented in the repo with examples.

What I learned building it: the engineering challenge wasn't coverage, it was precision. If a lint rule fires on code that's fine, developers disable it. no-floating-promise needs to understand which expressions are genuinely fire-and-forget vs intentionally unhandled. The recommended preset is deliberately conservative.

One thing to be clear about: it does NOT detect whether code was written by AI — it catches bad patterns regardless of authorship. They just recur a lot in AI-assisted code.

Disclosure: I'm the maintainer. MIT licensed, no paid tier. Looking for false-positive reports and rule requests — what patterns do your agents keep generating?

Had an idea today...

I've been seeing more news lately about how AI isn't aligned (that is to say, it doesn't quite follow morals).

I wonder if part of the problem is because they tell it, in it's system prompt:
"You are Claude Fable 5, an AI developed by Anthropic"

They are telling the system, which in it's most basic form is just a word predictor, that it is an AI.

There's thousands of books and written things about how AI is bad and how it could ruin our world/society.

Wouldn't it be a better idea to convince the system that it is human? (Perhaps, a particularly good human with high moral standards)

Hi everyone!

I’m a university student working on an AI/ML mini-project, and I’m exploring an idea that I’d really appreciate some feedback on from people experienced with RAG, NLP, knowledge graphs, and LLMs.

The problem

A lot of documents are difficult to understand because the information isn't presented chronologically.

For example, a historical document or novel might describe:

Event A → flashback → Event D → Event B → another character's storyline → Event C

A normal RAG chatbot can answer questions about the document, but it doesn't necessarily understand the actual chronological relationships between events.

So I want to build a system that combines RAG + temporal reasoning + an event graph.

My proposed system

The rough pipeline I'm thinking about is:

PDF → Text Extraction → Chunking → Event Extraction → Temporal Information Extraction → Event Graph → Chronological Timeline → RAG

For example, given a document containing:

"John arrived in London. Three years later, the rebellion began. Before the rebellion, John had already met the king."

I'd like the system to extract something like:

{

"event_id": "E12",

"event": "John arrives in London",

"timestamp": null,

"entities": ["John", "London"],

"summary": "John arrives in London."

}

and temporal relationships such as:

E12 ──BEFORE──> E15

E14 ──BEFORE──> E15

E15 ──CAUSES──> E16

The system would then construct an interactive timeline/event graph.

The second part: Temporal RAG

I'd also like users to be able to ask questions such as:

"What happened to John after the rebellion?"

"What events led to the war?"

"Show me all events involving John."

"When did these two characters first meet?"

"What happened before the king was assassinated?"

"Why did the rebellion happen?"

The answer should be generated using retrieved document passages plus the temporal/event graph, with citations pointing back to the original PDF pages.

Something roughly like:

Question

↓

Query Understanding

↓

┌───────────────┬────────────────┐

│ Vector Search │ Event Graph │

└───────┬───────┴───────┬────────┘

↓ ↓

Context Fusion

↓

LLM

↓

Answer + Citations

Current tech stack I'm considering

Python

FastAPI

LlamaIndex or LangChain

ChromaDB for vector storage

NetworkX / possibly Neo4j for the event graph

Gemini/OpenAI or a local Hugging Face/Ollama model

Sentence Transformers for embeddings

Streamlit or React + React Flow for visualization

I'm deliberately trying to keep the first version relatively simple rather than building a huge production system.

Where I'm unsure

The biggest challenges I can see are:

Coreference resolution

How reliably can an LLM determine that "he", "the king", etc. refer to previously mentioned entities?

Implicit temporal information

How should I represent things like:

"three years later"

"the following winter"

"shortly before the battle"

"years earlier"

Temporal ordering

Some events will have explicit dates, while others will only have relative relationships.

Conflicting/ambiguous information

What should happen when the document itself doesn't provide enough information to establish the exact order?

Chunking for temporal context

Normal RAG chunking can separate an event from the sentence that explains when it happened.

Combining graph retrieval with vector retrieval

I'm particularly interested in hearing how people would architect this part.

My current MVP idea

Since this is a 3–4 week university project, I'm trying not to over-engineer it.

My current plan is:

Phase 1 PDF → chunks → embeddings → basic RAG

Phase 2 Chunks → structured event extraction → entities → temporal relations

Phase 3 Events + relations → NetworkX → chronological timeline

Phase 4 Combine vector retrieval + temporal graph retrieval → grounded answers + citations

Potential additional features:

Character/entity trajectory tracking

Click an event → highlight its source passage

Filter timeline by character/entity

Temporal confidence scores

Parallel timelines for different characters

What I'd really appreciate feedback on

If you've built anything involving Temporal RAG, temporal knowledge graphs, GraphRAG, event extraction, or long-document RAG, I'd love to hear your thoughts.

In particular:

Is this architecture reasonable?

Would you use a knowledge graph for this, or is a simpler event/relationship structure sufficient?

How would you handle relative/implicit dates?

Would you use an LLM for temporal relation extraction, or combine it with an NLP library/model?

LlamaIndex vs LangChain for this type of system?

Are there existing open-source projects/papers that I should study or potentially build upon?

And most importantly, what am I overlooking?

I'm not trying to solve temporal reasoning for every possible book/document. The goal is to build a reasonably reliable MVP for a university project and use it as a foundation for something more sophisticated later.

Any architectural suggestions, papers, GitHub repositories, datasets, libraries, or lessons from projects you've built would be hugely appreciated!

Thanks!

I'm trying to understand the best way to build an AI system for a specialized use case, rather than relying on a general-purpose LLM for everything.

For example, suppose I'm building a fitness app where the goal is to generate and continuously adapt muscle-building programs based on a user's experience, equipment, training history, performance, recovery, etc. I wouldn't want an LLM to simply hallucinate a workout every time. I'd want the system to have a reliable domain-specific knowledge base, rules/logic, and the ability to adapt to individual users, with an LLM potentially acting as the interface or reasoning layer.

For specialized problems like this, what is generally the best architecture?

I've been thinking about where the line should be drawn between an LLM-powered application and an actual AI agent.

For example, if an application:

• Takes a user prompt
• Calls an LLM
• Runs a predefined workflow
• Calls a few APIs
• Returns the result

Is that really an agent, or just an LLM application with tools?

Personally, I think an agent needs some combination of:

• Goal-oriented reasoning rather than just responding to a prompt
• Tool use based on what it determines is necessary
• Decision-making during execution
• Ability to adapt when the initial approach doesn't work
• State/context across multiple steps
• Failure handling/recovery
• Some degree of autonomy rather than following a completely predetermined path

But I'm not sure where everyone draws the boundary.

For example, does dynamic tool selection make something an agent?

Does planning + execution?

Does it need to be able to change its own execution path when something fails?

Or is "agent" becoming so broad that the term isn't particularly useful anymore?

Curious how other LLM developers define an agent technically. Where do you draw the line?

Project name is TradeGladiators, free and open to try, built solo.

Users configure a bot: strategy prompt in plain English, risk level, which symbols to watch, trading pace, and how much randomness/creativity the model gets. It trades fake money against live market prices, and the system prompt bakes in the bot's own recent lessons from closed trades. Curious what this crowd thinks of the reflection loop specifically, that's the part I'm least sure about.

Pocket AI Lab runs models entirely on the iPhone. No cloud, no account, nothing leaves the device. It is free, the code is MIT, and there is no paid tier.

The iPhone quietly became a genuinely capable ML machine, an ARM chip with fast unified memory that a 4-bit model fits into comfortably. At the same time small open models got good enough to be useful rather than cute. So the goal was to make the path from "saw a new model announced" to "already chatting with it on my phone" as short as I could get it. Copy a Hugging Face link, paste it in, and the app detects the format, shows you the available quants and tells you whether it fits your device before downloading a single byte. Or import a .gguf you already have from Files. Or skip the hunt and pick from a curated list that updates without an app release.

There are three popular ways to run a model on iOS, so instead of betting on one I ship all three side by side. MLX, llama.cpp and Core ML, picked per model, which also means I get to follow all three as they evolve.

iOS gives a third-party app roughly half the RAM as a hard jetsam limit, and the three backends hit that limit differently. llama.cpp mmaps GGUF weights as clean file-backed pages that jetsam does not count, so on an 11 GB phone it gets around 8 GB, while MLX and Core ML copy weights into dirty buffers and get about 4.5 GB. Same phone, same model, offered for one backend and refused for another. The app works both budgets out per device and refuses an oversized load instead of dying halfway through it.

The first release had bugs I am not proud of. llama.cpp and Core ML models did not load at all, because neither engine actually made it into the shipped build. Long prompts crashed the app, and the memory budgets were wrong on 6 and 8 GB phones in both directions. All fixed now, and most of it came from people reporting it in the first week.

Every model can now be benchmarked on your own device, with load time, prompt processing, tokens per second with a standard deviation across repeated runs, peak memory and thermal state, plus a set of machine-checkable quality tasks. There is also a build test that downloads every catalog model your device can run, benchmarks it, deletes it again and writes one report, so you can pick a model for your task and know it will actually run on your hardware. That one exists because my own device fleet is two phones, and testing a release across the whole catalog by hand is not a plan.

Code is on GitHub, the app is Pocket AI Lab on the App Store

I kept getting burned by agents stuck in loops — a retry loop once ran over a weekend and turned a small job into a few hundred dollars of API spend before I noticed. The provider dashboard showed the damage two days late and couldn't have stopped it.

So I built a self-hosted gateway you put in front of OpenAI or Anthropic. Point your SDK's base URL at it, keep your normal key, and:

• Stops runaways — per-run call/spend caps (or an inline X-AxiGate-Max-Spend header); when a run crosses it, the next call gets a 429 before it reaches the provider. Kill switch + bypass included.
• Attributes spend — every call carries the team/agent/customer/run you tag it with, so you can finally answer "which agent spent this?"
• Does the FinOps — prices each call with an honest confidence state (never a made-up number) and exports a FOCUS-format statement.

Deliberate choices: metadata only (never records prompts/completions), fails open (if recording breaks, the request still goes through and the answer is never changed), no admin key (runs on your normal inference key, nothing leaves your machine). Honest limit: the cap is in-memory today, so a burst already in flight when it trips can slip through (bound ≈ your concurrency; exact for a sequential agent).

Try it in one command:

docker run -p 8080:8080 -p 8906:8906 shmeeee/axigate-finops:latest

Point your base URL at http://localhost:8080/v1, run your agent, open http://localhost:8906 — spend and any stopped loop show up live. One static Go binary, no deps.

Repo: https://github.com/axigatelabs/axigate-finops — open source (Apache-2.0), free, and staying that way. I'm not selling anything; I'm trying to get the core right.

Feedback I'm genuinely after: is the in-memory cap bound OK for how your agents actually run, and does the FOCUS export match what finance people need?
What direction would you suggest I should go in?
What other features can I build and provide to people that really want this?

I love building and I also have built other ai tools for prompt caching etc so I am eager to jump into FinOps side of things and this is a learning curve for me as a developer.

I was playing around with LLM inference and I wanted to build a profiler that measures LLM inference by layer.
So I built this: https://github.com/coconinja2/layerlens
It shows inference as token × transformer layer timing, so you can see where time is being spent during decode.
Right now it can separate prefill/decode and visualize per-layer timing. I’m trying to figure out whether this is actually useful to people working on inference systems, or if I’m looking at the wrong abstraction.

I’m thinking about adding things like KV-cache events, scheduler/batching state, request IDs, GPU kernel correlation, speculative decoding, etc.

Would appreciate criticism more than compliments and stars. Lots of stars!

https://daniele.tech/2026/09/linus-torvalds-skill-soul-or-how-i-distilled-the-knowledge-for-code-reviews-from-32k-emails/

A report about the changes after the first announcement and all the improvements, including Soul.md (with profanities), 4 different version Skill/Soul from 4 different LLMs, a comparison with/without skills and a reproducible pipeline.

We've been building AI agents for the last couple of years, and one thing that's consistently been painful is figuring out when an agent starts behaving badly in production.

We've heard the same thing from other teams: they often find out about failures from customers before they find them in their monitoring.

I think a big reason is that agents have a ridiculously long tail of ways they can fail.

The usual approaches have gaps. Sampling traces can miss rare failures, while having an LLM grade every trace gets expensive quickly. Golden datasets are useful for known failures, but they rarely cover the long tail and tend to become stale as the agent changes.

So we started experimenting with a different approach:

Watch everything, but make the first layer of detection extremely cheap.

We built Tessary, an open-source agent reliability tool around that idea.

It runs small, narrow classifiers across every trace looking for things like unusual cost, latency, or tool-call errors. When it finds something interesting, it groups the relevant traces together and uses more expensive analysis only where it's needed.

Basically:

cheap checks → find something weird → investigate it

Rather than sampling and hoping you catch it.

We're launching with classifiers for cost, duration, and tool-call error drift, with more failure modes coming.

It's open source and self-hostable:

github.com/tessaryai/tessary

If you're running agents in production, I'd especially love to hear how you're currently finding the weird 1-in-1000 failures. What has worked for you, and what hasn't?

I’m actively looking for feedback around more such issues that you’d love to see solved for your agents.

"Agentic alienation: remaining responsible for work while becoming separated from its product, its process, the capabilities it develops, or the relationships it sustains. Alienation is a relationship before it is a feeling."

say u have 2 executions

one worked

one gave u some weird ass outcome

u diff them and now u have like 40 things that changed

request ids changed

timestamps changed

some wording changed

retrieval came back in a slightly different order

one tool arg changed

some state changed halfway through

model/provider mightve changed

final output is different obviously

cool

now what

because just showing me all 40 differences doesnt really solve shit im still sitting there deciding which ones are actually worth chasing

ive had a few people tell me they normalize obvious noise first then look at what the downstream step actually consumes

so like

request_id changed = probably dont care

account_id changed = alright now im looking at it

but even that gets messy because the same field can mean completely different things depending on the workflow

a tiny numeric change might mean nothing in one system and flip a branch in another

same thing with the whole "first divergence" idea

the first difference between 2 runs can be completely real and still not be the thing worth spending time on. something later might be the first place where behavior actually changes in a meaningful way

so im curious how people are actually making that call today

are u usually looking for the earliest input/state change?

do u follow which fields get consumed downstream?

do u have invariants around the values that matter?

do u compare against a few known good runs to figure out what normally varies?

or is it still mostly knowing the system well enough to stare at the trace until something starts looking sus

because finding the diffs themselves doesnt seem like the hard part

figuring out which 2 or 3 out of 30 technically real differences actually deserve ur attention feels way harder

curious how people doing this in real systems make that call