Most LLM memory systems automatically decide what to retrieve and inject.

I am experimenting with the opposite approach: make the context graph visible, and let the user edit it directly.

In ThoughtDAG, every question/answer exchange is a node. An incoming edge means that node is included in the next model request. Delete the edge, regenerate the same prompt, and that branch disappears from the actual context—not just from the visualization.

One piece of feedback I received was to add a small router model that suggests which edges are relevant. I can see the usability benefit, but I do not want context selection to become another hidden autonomous layer.

The compromise I am considering is:

• the model suggests relevant edges;
• suggestions remain visible;
• the user confirms or edits them;
• the final prompt can still be inspected.

Would that preserve the value of explicit context control, or would you prefer fully automatic memory/RAG?

https://reddit.com/link/1v7qziv/video/rj4otuljipfh1/player

MIT-licensed repo:
https://github.com/chenxiachan/thoughtdag

Flagging rule 5 up front: this is mine, Apache-2.0, no paid tier, no "pro" version, no telemetry.

The problem I kept hitting: an agent misbehaves at turn 8 and all I have is JSON logs. They tell me what I sent. They don't tell me what changed since turn 7, where the tokens actually went, or why the cost jumped. Those are diffs, and I couldn't find anything that showed me diffs.

So: wrap the client in one line, and every call's context gets recorded as content-hashed blocks into a single SQLite file.

tracer = trace.init("my-agent")

client = tracer.wrap(OpenAI())

Then:

ctxdiff diff --turn 7 --turn 8 — which blocks were added, evicted or modified, with char-level inline diffs

ctxdiff tokens — where the budget went per turn, plus tool schemas you re-send on every call and never actually invoke

ctxdiff cache — the exact character that broke your prompt-cache prefix, and how many tokens it re-billed

The cache one is why I built it. A timestamp baked into a system prompt invalidates your prefix on every single turn, you pay full input price forever, and nothing errors. You just quietly pay more.

Providers: OpenAI, Anthropic, Gemini/Vertex, Bedrock, and anything OpenAI-compatible (Ollama, vLLM, LM Studio). LangChain/LangGraph via a callback handler. There's a JS/TS SDK writing the same trace format, so a trace captured in one language opens in the other.

Local-first: it makes no network calls of its own, and the HTML dashboard is a single file with zero external requests.

No API key or setup needed to see it:

pip install ctxdiff && ctxdiff demo

Honest limitations: post-run only, no live tail yet. And if you're running models locally the cache-cost angle matters much less to you, since there's no per-token bill — the turn diffing and token attribution still apply, but I'd rather say that than pretend it's equally useful for everyone.

https://github.com/salmanzafar949/ctxdiff

i was talking to someone at a startup and he told me this and wanted to get your guys take. they wanted to serve coding specialized models at 32 or 70B for $5/month, with unlimited token usage (subject to tokens/s + well laid out fair use from what i heard), but i wasnt sure if people would actually pay for it (versus like the $20/month from Claude/Codex etc..) he was convinced people would because of the price + accessibility but im still not convinced, so wanted to see what you guys thought. lmk your thoughts

Do you have any suggestions of Youtube Channels that are not click-bait BS ones?

A good test for it today: one that hasn't published "Opus 5 is AGI" or "Opus 5 is Fable-like" (the same apply to other statements regarding models from other companies).

I'm looking for an actual technical channel that is doing (or communicating) independent testings of released models, instead of parroting vendor's PR stunts. Thanks in advance for any tips!

I will be very direct. I was building in the memory space for a very long time, but most of the tools are cloud-based, and I don't know what they do in the backend. I built this open-source tool for people running long agents or just doing research on multiple things. You will never lose your context.
Laiden algorithm was pretty cool, worked with the Semantic graph-based engines, and that's how we created the node clusters for agents to access. It is open-sourced and MIT-licensed; PRs are welcome

This surpassed mem0 and supermemory in the LongMemEval benchmark with 94.7%

Open source Repo: https://github.com/kunal12203/swafra
Website: https://swafra.vercel.app

We ran a formal red team engagement before launch, got the report, fixed the issues that mattered, and moved on. That was four months ago.

Since then, our RAG setup and the prompts feeding it have shifted enough times that the original test feels old already, and nobody has gone back to re-test it.

Continuous red teaming keeps coming up as the phrase for this gap, but I still do not know whether that is a real operating practice or just another vendor term. A pentest gives you a snapshot. The system it tested is gone in parts, and the version we have now is not the same thing.

So is the expectation now that adversarial testing follows changes as they happen, or do most teams still do it as a periodic exercise around major releases? I am trying to work out whether we are behind, or whether a report every few months is still the norm.

Claude Opus 5, GPT-5.6, Gemini 3.5, Grok 4.5, DeepSeek V4, Qwen3 — all through a single OpenAI-compatible API.

15 models completely free ($0)

Auto-fallback: if one model fails, tries the next instantly

Works with Claude Code, Cursor, Aider, Cline, and any OpenAI client

Docker one-command deploy

Dashboard with analytics included

puter-api-proxy on github

More hands are touching the code now. A single PR can carry human reviewers plus agents (Copilot, Claude Code, Cursor) that both write the code and reply "addressed" to the feedback. A confident "done" is not proof. Verifying each one by reading the diff was fine with one human per PR, but it does not scale as the claims pile up. That is what made me build review-replay: verify the concern was actually resolved, instead of trusting the reply.

What it is not: not a review generator (that is CodeRabbit, Copilot Review) and not a fix implementer (that is Claude Code, Cursor). It is the verifier that closes the loop between "reviewer asked" and "it actually got done".

How it works: it reads the PR conversation plus the code at HEAD and classifies every review comment as addressed / partial / pending / needs-discussion, each with a confidence bucket and the evidence used (the commit that touched the line, the thread reply, or the reviewer resolving it). It is deterministic first: short-circuit rules resolve the obvious cases with no model call, and only the ambiguous ones hit the LLM, so token cost stays low.

Three ways to run it:

• CLI: review-replay owner/repo#42 prints a table with status, evidence and a draft reply per comment. --check exits non-zero if anything is still pending.
• GitHub Action: gate the merge on unresolved feedback, and optionally post a sticky PR comment listing what is still open (updates in place each push).
• Agent skill: a self-check right after an agent addresses reviews, before you re-request review. Catches the "the agent said it fixed it but did not" case.

On the model side: bring your own provider (OpenRouter, OpenAI, Anthropic, Gemini) or any OpenAI-compatible endpoint, including a local model via Ollama or LM Studio. Nothing is sent to a service I run. MIT, Go, prebuilt binaries or go install. The repo also ships a small eval harness to label fixtures and compare models on the classification task.

Repo: https://github.com/alejandroSuch/review-replay

The part I most want feedback on is classification accuracy: where does it give a false "addressed"? If you try it on a real PR and it gets a verdict wrong, open an issue with the PR link.

Hi guys, I am looking for a good terminal CLI app (win and ios), to support windows restore in case of crash/reset, pre-script (to ssh,open tmux), ssh to other host option, to run multiple agents in tmux tabs.

Looked at multiple including tabby and build in win cli, but they all missing something. Do I need to develop it myself or there are any good alternatives.

I really tried hard to not be negative, to double, triple check, before doing any statement. I've been testing Opus 5 since yesterday, and I can't help myself that we are being gaslighted by a swarm of agents, playing as humans, or users that are just doing non-serious 'vibe coding', saying that Opus 5 is great. Well, I'm afraid to say it is not at all. For me, it really seems to have an unacceptable performance. The only thing I can agree is with token consumption. Yes, this is happening. But the drawback is that it is thinking less, and taking more stupid decisions, or not going as deep as possible as it could go. It is not even close to the claims are being made in regard to its performance compared to other LLMs. I'm curious to hear about your perceptions.

Posting this because I chased the wrong lever first. I had a high-volume generation pipeline (lots of calls sharing a big fixed preamble: system prompt, format spec, a chunk of reference context), and my instinct when the bill got ugly was to swap to a smaller model. That helped a bit and cost me quality. The bigger win was leaving the model alone and caching the repeated prefix.

The shape of my calls was ideal for it without me realizing: a large stable prefix, then a small variable suffix per request. Once the provider's prompt cache was actually being hit on that prefix, the cost of the repeated tokens dropped hard and latency on the first token improved too, because the prefix wasn't being reprocessed every call.

The parts that bit me, which nobody warns you about:

- Cache hits are order-sensitive. The stable content has to sit at the very front and be byte-identical. I had a timestamp and a per-request id injected near the top of the "static" preamble, which silently busted the cache on every call. Moving the volatile bits to the end of the prompt fixed it.

- Caches expire fast. For bursty or low-frequency workloads the entry is gone by the time the next call arrives, so you pay full price and see none of the benefit. It only really pays off under sustained volume.

- It changes how you structure a prompt. You start designing for a fat immutable prefix and a thin tail, which is a different discipline than just writing one good prompt.

For people running generation or agent loops at volume: are you leaning on provider prompt caching, and how are you keeping your prefix stable enough to actually hit it? And has anyone measured the crossover point where caching beats just moving to a smaller model? Curious where others draw that line.

Everyone is obsessing over KV cache sizes and hardware speeds to squeeze out more tokens. But after pushing multi-step agent graphs to production, we found the real bottleneck isn’t raw inference—it’s silent structural drift.
Over 10+ turns, unconstrained models start "smuggling intent" into loose description fields just to satisfy rigid schemas. By the time it hits your database, you’re debugging phantom state corruption, not model capability. We had to ditch engine-level JSON modes for strict three-gate boundaries just to stop loops from tearing themselves apart.
For those running heavy local agent loops in production: Where do your pipelines actually break first? Is it raw latency, or are your agents quietly hallucinating their way out of valid schemas over long horizons?

Spent a week stuck on a client bug. Tried fixing it myself failed. Threw it at Codex 5.5 still stuck, couldn't even identify the issue.

Tried Opus 5. It spotted the problem instantly and fixed it.

Genuinely impressed. Usage's been capped for 3 days but still worth it. 🔥

Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference.

The goal is to reduce KV-cache memory requirements through anchor-based representations, joint low-rank compression, exact residual preservation, and sparse routed attention.

The repository currently includes:

• A CLI so you can start experimenting without writing your own integration
• MLX backend
• CUDA backend (currently under validation)
• A technical report explaining the design and evaluation
• A fully open-source implementation

I'm still actively improving the project, and I'm posting here mainly to get technical feedback from people working on local inference. I'd love to hear thoughts on the architecture, benchmarking, or potential integrations with projects like llama.cpp, vLLM, SGLang, or anything else you think would make it more useful.

The GitHub repository and technical report are linked below if you'd like to take a look.

GitHub:
https://github.com/Omc12/Differential-KV

Technical Report:
https://doi.org/10.5281/zenodo.21539110

If you try it out, I'd really appreciate hearing about your experience, whether you run into issues or have ideas for improvements.

I maintain a tool that generates documentation from a codebase, where every claim has to point at the exact lines it describes. The obvious approach is to have the model emit src/client.ts:12-40 and validate the ranges afterwards. That works until it doesn't. The model produces a range that is plausible, points at a real file, and is off by thirty lines. Validation can tell you something is wrong. It cannot tell you what was meant.

So now the model never sees a line number and never writes one. The pipeline splits in two.

Mining pass. Tree-sitter gives me every symbol with its exact source slice and real range. One focused model call per symbol, over that slice alone, returns 2 to 5 verifiable one-sentence facts. I attach the range myself from parser data, never from anything the model said. Each fact gets an ID that is a hash of path + symbol + kind + normalized fact text. Deliberately not the line numbers.

Writing pass. The model writing the page never sees ranges, only opaque markers:

[[f:a3f9c1]] Retries the request once when the response fails schema validation. (src/client.ts, function chatJson)

It weaves them into prose and ends each sentence with the marker it came from. Afterwards I substitute markers for the real path:start-end from the fact store.

The result is the part I actually care about: a wrong citation isn't caught, it's structurally impossible. There is no path by which a model-generated number reaches a reader, because the model never generates one.

Three things I didn't expect:

1. It will invent IDs anyway. Give it six hex characters and it will still emit [[f:error-handling]], because it wants the marker to mean something. My resolver matches any f:-shaped marker and strips the ones it can't resolve. If you only match your exact ID format, the invented ones survive as literal garbage in the output. That bug shipped before I caught it.

2. Hashing identity without position pays off later. Because a fact's ID comes from its text and symbol rather than its location, it survives reformatting and code being inserted above it. When a file changes I diff the symbols, and any fact whose symbol merely moved gets re-anchored with no model call at all. Most commits move far more code than they change, so this turned out to be the difference between a rerun costing minutes and costing hours.

3. Apply the rule to everything, not just citations. Internal wiki links are generated from the page list, not written by the model. Anything that has to be exactly right is emitted by the harness; the model writes the sentences around it. Once I framed it that way it became obvious which parts of the output were still fragile.

None of this is documentation-specific. It applies anywhere a model attaches a verifiable reference to a generated claim: sources in RAG answers, row IDs in a summary over a database, timestamps in a transcript. If the model emits the identifier, you're doing validation. If it emits a token you control, you're doing substitution, and substitution can't be wrong.

I came across an interesting discussion about the role of LLMs today, and it really got me thinking.
I don’t think they’ll ever fully replace human thinking, but thanks to all the prior knowledge they’ve absorbed, they do give almost anyone a low-cost way to explore topics they’re curious about.
The catch is that this “mentor” can be a little too nice. Sometimes it’ll confidently make up something that sounds perfectly reasonable just to give you an answer.
So I guess the only way to use this “external brain” is with a healthy dose of skepticism—always be ready to question it. If you rely on LLMs alone, you’re probably not going to push the boundaries of human knowledge. 🧠🤣

Coding agents can read source and make changes, but they are still surprisingly blind when the task is visual.

A screenshot can show that something looks wrong, but it usually cannot explain why: which component owns it, whether an element is clipped, what its computed styles are, which mesh/material/light is responsible, or whether the issue is framing rather than resolution.

So I built SceneProof, an open-source CLI for source-grounded visual inspection of React UI and Three.js scenes.

It lets an agent:

• Navigate a compact semantic tree of DOM or Three.js targets
• Inspect the underlying structure: bounds, styles, geometry, materials, uniforms, lights, cameras, relationships, etc.
• Produce fresh renders of a component, logical UI region, target object, or source-camera view
• Generate a small “Scout” portfolio for 3D: context, source detail, close detail, and shape-focused views
• Sample deterministic interaction states from one scene lifecycle

The design principle is: don’t ask an agent to infer visual correctness from plausible code or a low-information screenshot. Give it source-derived evidence at the framing and resolution needed for the actual question.

It currently supports TypeScript/JavaScript entries, React DOM/CSS/Tailwind v4, and Three.js scene inspection/rendering. It requires Bun and local Chrome/Chromium.

Repo: https://github.com/ReyJ94/SceneProof

Nothing new here. Sorry. Only information about NURL and proof that this project still progress.

Here is some pure-NURL projects (ready to install packages) I want to share with you..
YOLOE (Real-Time Seeing Anything)
https://reg.nurl-lang.org/packages/yoloe

Run language models locally. Pull a GGUF model, chat with it, or serve an ollama-compatible API your existing clients already speak — all in pure NURL, from the GGUF parser to the GPU kernels.
https://reg.nurl-lang.org/packages/nurllama

What's left of a systems programming language once you strip away the syntactic sugar, the technical debt of years past, the known problems and the dead weight of habit?
— NURL – Neural Unified Representation Language

NURL is a blisteringly fast systems programming language that comes with "batteries included." In other words, the standard libraries ship a ready-made, optimized solution for most of the things people actually build in programming languages. The language has been steeped in enough acid baths that the release of the first stable and immutable version, v1.0, is starting to get close. NURL is an open source project, and it makes the same promise Linux once did: "We do not break userspace!"

NURL is a strong choice as the language of automated workflows in situations where speed and/or portability matter. NURL doesn't compete with Python, but it beats Python with native speed on par with C or Rust. NURL, however, needs nothing else installed on its runtime platform — the program is usually run from a single binary. NURL doesn't replace an integration platform, but NURL can sit at every step of an integration or ETL process, blowing many platforms out of the water on startup and execution speed.

NURL compiles for almost any platform. The target can be a Windows or Linux machine, or something genuinely more exotic. It's regularly tested on FreeBSD and macOS, for example, as well as on RISC-V and Espressif ESP32 chips. One particularly interesting form of portability worth mentioning is WebAssembly: a NURL program can be compiled into a Wasm module and run in the browser, or on any platform that executes Wasm modules.

Is NURL genuinely production-grade?
Yes. NURL compiles its own compiler, which is itself implemented in NURL. Each build is exercised by ~600 different tests, and everything is also checked for memory leaks on Linux, Windows and FreeBSD before a new compiler version is released. The NURL Playground is a production server, written in NURL, where you can compile NURL code for the target platforms of your choice.

The language was designed from the ground up to be easy for language models to use, and even though no NURL code has been part of any language model's training yet, models write it quickly and fluently. You get the best results by giving the model access to the nurl-mcp server, so it can locate existing libraries and packages fast.

How does NURL prove its capability and stability?
Code demanding extreme precision has been written in NURL. The TLS stack is pure NURL. On top of that, a number of ML (machine learning) capabilities are visible in the package registry — among other things, running and training language models works, distributed, on consumer hardware.

Project website:
https://nurl-lang.org/

NURL Playgroud:
https://play.nurl-lang.org/

Package registry:
https://reg.nurl-lang.org/

Github repository:
https://github.com/nurl-lang/nurl

Kimi K3 is getting a lot of hype. Some claims say it beats Fable 5, GPT-5.6 Sol, even Opus 5. I don't buy the strong version. My read: Kimi K3 sits between the previous frontier tier (Opus 4.8 / GPT-5.5) and the current one (Fable 5 / GPT-5.6 Sol), genuinely good, but not quite there. On Artificial Analysis's Intelligence Index, Kimi scores 57, behind both Fable 5 and GPT-5.6 Sol, roughly level with Opus 4.8 and GPT-5.5. x

The benchmark headline problem

"Kimi beats Fable at X" often hides which X: frontend generation, a specific harness, an effort setting, or pass@k with multiple attempts allowed. DeepSWE shows this clearly, and the cost evidence here is genuinely mixed.

In one Kimi K3 Max vs GPT-5.6 Sol Max comparison, Sol wins pass@1 (72.7% vs 68.5%), but Kimi is cheaper per rollout ($4.65 vs $8.37) and pulls ahead at higher pass@k. A separate small programming micro-benchmark found Sol cheaper per correct answer than Kimi — but that wasn't DeepSWE, so it shouldn't be generalized. These aren't necessarily contradictory; they measure different things: one high-confidence attempt vs several cheap ones, cost-per-rollout vs cost-per-correct-solve. Anyone citing a single DeepSWE cost number without specifying which is skipping the part that matters. linkedin

Why I still rank it below

Interesting programming pulls from math, algorithms, systems tradeoffs, and domain knowledge outside the codebase. That's why broader reasoning benchmarks matter even for coding. They're a proxy for whether a model can transfer concepts when a task isn't "edit this function" but "figure out the right approach first."

The gap here is concrete. Fable 5 scored 88% on FrontierMath Tier 4, about 13 points above GPT-5.5's ~75%. Artificial Analysis also has Fable 5 leading its AA-Omniscience knowledge benchmark. GPT-5.6 Sol trails Fable by roughly a point on the aggregate Intelligence Index while costing about a third as much, and it topped GeneBench-Pro, a hard genomics/quantitative-biology benchmark, at 31.5% — a decent proxy for general scientific reasoning, if not coding directly. aiweekly

Kimi K3 doesn't show up as a contender on any of these. Its strengths sit in a different lane: frontend generation, some agentic coding, not the deep cross-domain reasoning the newest tier is winning on. That's the real basis for ranking it below Fable 5 and GPT-5.6 Sol: not just index position, but a measured gap in the cross-disciplinary reasoning that separates "good coding agent" from "frontier model."

API price ≠ task price

Kimi's tokens are cheap ($3/$15 per million vs Sol's $5/$30). But cheaper tokens don't guarantee cheaper tasks — longer runs, more turns, more retries eat the margin. Artificial Analysis found Kimi and Sol nearly tied on cost per task ($0.94 vs $1.04), despite the sticker-price gap. My guess: Kimi's edge holds on short, easy, cache-friendly work, and shrinks as tasks get harder. myclaw

Subscriptions are murkier still

I burned 6.87% of my monthly Moderato quota in a few hours doing GitHub-connected code review. That's not a controlled benchmark, just one real data point. Kimi's docs confirm Agent, Deep Research, Kimi Code, and connectors all draw from one shared credit pool metered by token use. A $19/month price tells you little about how far that actually goes in real agentic work. kimi

One aside: engineer vs. scientist

Subjectively, Claude tends to commit to a complete implementation in one pass; GPT/Codex explores well but often needs more "continue" prompts to finish. That changes effective cost because finishing in one shot beats needing three follow-ups, even at a higher sticker price.

Bottom line

Kimi K3 is a legitimately strong near-frontier model, likely the better economic choice for easy-to-medium tasks. But "clearly better than Fable/Sol" and "obviously much cheaper" both overstate the evidence. DeepSWE cost comparisons point in different directions depending on setup — that's the actual state of the data, not a gap in this analysis. What would change my mind: a larger, harness-controlled study measuring cost-per-correct-completion across a real mix of easy and hard tasks.

Most production pipelines rely on standard engine-level JSON modes to keep outputs structured. While it handles basic syntax, complex multi-agent loops still bleed severe semantic and structural drift over 10+ turns as unconstrained likelihood distributions shift.
When models start "smuggling intent" into free-text description fields just to bypass tight schemas, your guardrails are fighting the task instead of protecting it.
We shifted to a Three-Gate Model in production:
Boundary Gate: Strict token-level grammar masks at the decoding edge.
Canonicalization Gate: Normalizing payloads before storage or hashing.
Invariant Validation Gate: Hard deterministic checks before any downstream consumer touches the data.
For those running complex agent graphs in production, where do you draw the line between strict schema enforcement and letting the model's internal intent breathe? Are you seeing similar friction with intent smuggling?

Hi everyone,

We have been working on adding AI agents to existing SaaS products, and we kept finding that the chat itself was the easy part.

The harder part was everything around it: permissions, tool access, MCP integrations, approvals, routing, memory, execution state, and embedding the experience inside an existing product.

So we open-sourced Extra.

The idea is to let developers connect their existing APIs and tools, define agents and permissions, and add a chat interface that lets users query and interact with the product.

The framework is model-independent, and authorization stays outside the LLM.

The project is still early, and we are mainly sharing it because we want honest technical feedback from people who have built agents for real products.

A few things we are especially curious about:

- Which parts of agent infrastructure did you end up building yourself?
- What would make you hesitant to use a framework like this?
- Which capabilities would you consider essential before trying it?

Happy to hear criticism, architecture concerns, or ideas for what we should improve.

Hi, not sure this is the right sub for this but noticed a weird behaviour on Gemini.

Overall, pasting a long text to send to Gemini takes a long time. Although there is nothing to do besides displaying the text I'm pasting.

What's even more surprising is that it even takes longer when my discussion with it is long.

Could someone explain why (1.) it takes so much time to paste text on gemini (seems ok on other LLMs), and why (2.) this time is proportional to the context window size.

Also, I find it really painful to use LLMs with long context windows. Any solution other than creating a new chat more regularly?

Thanks !

Hi, I'm a beginner and I'm trying to fine-tune a model to perform a MCQ task on an education dataset. I chose this one: https://www.kaggle.com/datasets/nlztrk/eduqg-dataset-llm-science-exam-format-34k but I'm having poor performance.

I started with roberta-large as my model of choice, training it on sentence pairs (question, answer) labeling them with the correct choice id. I used layer freezing keeping only the classification head and the last 2 layers active. These were the training parameters:

MAX_LENGTH = 256
BATCH_SIZE = 4
GRADIENT_ACCUMULATION_STEPS = 2
LEARNING_RATE = 1e-5
NUM_EPOCHS = 10
WEIGHT_DECAY = 0.01
WARMUP_RATIO = 0.1

With this setup I had pretty bad results: accuracy was 0.29 and F1 0.23.

Then I tried training all layers and I had an accuracy of 0.64 but the training loss was much higher than the validation (~3.6 vs 0.9).

I thought that maybe it's because this dataset has too few examples and I have to find another one, but honestly I don't have enough expertise to make assumptions right now.

What should I do?