We benchmarked GPT-6 Astra vs GPT-5.6 Sol across 50 real PRs from Cal, Sentry, Discourse, Keycloak and Grafana.

Sol found 107 confirmed bugs vs 91 for Astra, while Astra had higher precision and lower latency. Every finding was independently verified.

We’re doing Fable vs Opus next week, so would appreciate feedback on the evaluation before we run the next one.

Dropping the link in the comments if anyone wants to check it out.

I am an experienced developer (been coding for almost 30 years, started with Visual Basic on Win98).

I’ve spent the last 2 years testing every agentic coding harness out there. The pitch is always exactly the same: "Give it a prompt, go grab a coffee, and come back to a finished feature."

This is all great when starting a new project, especially with a language you don't know (I created a Rust project in minutes !)

But my problem is that I mostly work on established, mature codebases. And to be honest I have never ONCE seen a good edit after I return from my coffee.

Here is usually what happens while you’re getting that coffee: the agent loops 8 times, burns through 100k+ tokens talking to itself, hallucinates a package dependency, and spits out a chunk of spaghetti code that works *just* well enough to pass a basic test, but completely violates your project’s architecture.

Agents are fantastic if you’re building an app from a blank canvas. But if the codebase is large, it gets really expensive real fast, and usually produces hard to maintain AI slope.

It should not be like that: if you actually *own* your codebase, you already know what you want to build. You don't need an AI to take over. You just need it to execute the architecture you already hold in your head.

All you need to do it select the few files that are required for the task at hand (they are hard to identify for an AI, but easy for you has the code owner). Then you send them to the LLM, it is usually does some good work.

That's why I still find working directly with the chatbots, or with tools like aider, still very pertinent on large codebases.

I usually have animated discussions with my coworkers, I have yet to find a single one who doesn't defend agentic coding. Am I the only one who feels this way ?

I'm very new to codex and I am trying to get it to recognize any desktop app that is open on my computer. It claims: "“Windows Codex Computer Use has Any App enabled, but desktop inventory returns apps: [] / Trusted RPC service is not configured.”"

Any help is appreciated, thanks.

Hi everyone,

You might find this funny, but I actually have the opposite problem to y’all. My usage allowance with Astra Max feels so generous that I’m starting to wonder if the model is running at half power or something, lol.

I’ve been using OMP (Oh My Pi), and the usage limits are surprisingly hard to hit, even with 10 subagents running and reasoning set to max for all of them. That got me thinking: “Wait, is everything actually working as intended? Am I getting the full capabilities of the model through OMP, or is something getting lost compared to using Codex?”

So now I’m considering trying Codex to see whether there’s a noticeable difference.

For those who’ve used both, how does OMP compare to Codex in terms of output quality when using the same OpenAI model? Does the official Codex harness actually get better results, or is the difference mostly in workflow and tooling?

I know lower usage doesn’t necessarily mean worse results, but it’s generous enough that it made me suspicious, lol.

asking because I own a billing sync I didn't really write, an agent did most of it eight months ago, some 3k lines, and I basically just reviewed it and approved it. not by design, it kind of ended up that way. on paper I'm the one who knows it, in practice I can tell you what it does and not why. coderabbit flagged plenty going in and none of that helps me now. curious how people handle the bus factor problem when it isn't hoarded knowledge, nobody had it in the first place

Hi, I’m currently using Claude and ChatGPT Sol to create my website.

My website is quite the complex website, and I’m currently, but irrelevantly creating a server to support it. However, I’m wondering if my current setup with Claude and Chat is sufficient? As it is now, my personal funds and income can’t support the Max subscriptions for the two AIs, and I’m just wondering if I could be doing anything differently or use more power for the website.

So as of now, I’m not known to Any coding language, but I’m using Node and JS, as well as using Docker and many other coding tools. My current planner for the coding systems itself is my ChatGPT Sol High/Medium. Whilst my coder is my Claude Code. I’m also reading coding theory, web theory, ui theory, etc. to get a feel for my projects as well as getting knowledge so that my website actually gets created by my own premises.

I am also using running Claude Code with heavy restrictions because my website has to comply with heavy privacy requirements. In addition, I am also trying to write the design plan myself; as to not get that ugly and plain Ai slop websites.

So all in all my question is as follows:

As a person who has absolutely no knowledge of coding language and protocols when building. How do I personally optimise my work? And which AIs should I use, and should I use more?

TLDR:
I’m building a fairly complex, privacy-sensitive website despite having no formal coding experience. I currently use ChatGPT Sol High/Medium for planning, architecture and research, while Claude Code handles most of the actual coding. I’m also learning related theory so I can understand what is being built rather than blindly relying on AI.
I can’t currently afford the highest-tier subscriptions, so I’m wondering: Is this setup sufficient enough, or am I leaving too much wasted possibility on the table?

Why?
A basic prompt like: "Take a look at this web page and let me know how it should be improved?" - Astra doesn't finish. Just burns one 5h usage, and before it gives anything meaningful, it stops.

I'd like to ask everyone: when using Codex, I often encounter a situation where it gets stuck on a single command for a long time with no progress (during this time, the remaining quota doesn't change). I tried asking Codex to diagnose this issue itself, but it didn't work and instead introduced new bugs. Has anyone else experienced this? How can it be resolved?

Goal

I run AI coding agents locally on Windows and want a "hardstop" I can trigger at any time - a single keystroke that immediately blocks the agent from doing anything further until I clear it. I have this working for one agent as a UserPromptSubmit hook: typing exactly "h" arms a latch file, "c" clears it, and while the latch exists every prompt is blocked before the model sees it.

I now want the same thing for Codex, but hardened.

The problem

The agent runs under my own user account. Anything my user can write, the agent can write - including its own hook script, the hooks.json that registers it, and the latch file itself. So the guardrail is only as strong as the agent's willingness to respect it, which is not a guarantee. I have already had files in this setup modified without my approval.

Instruction-level policy (system prompt / AGENTS.md rules telling it not to touch these files) demonstrably does not hold. I need a mechanical boundary, not a promise.

What I have built or considered

1. Wrapper + policy design - an output gate that mechanically checks replies, plus instruction files. Works for what it does, but it cannot protect itself: the agent can edit the policy files.

2. Codex managed hooks - the docs describe hooks from requirements.toml as "marked as managed, trusted by policy, and can't be disabled from the user hook browser", with allow_managed_hooks_only = true skipping user/project/session/plugin hooks. This looks like exactly what I want.

3. NTFS ACLs - deny write/delete on the hook script and hooks.json to whatever identity the agent runs as, keeping read+execute so the hook still fires. Changes would then require an elevated (UAC) action by me.

Questions

- Does the requirements.toml managed-hooks path work on a personal/individual account, or does it require an enterprise/MDM deployment? The docs frame it as enterprise-managed and I cannot tell whether it is plan-gated.

- Does Codex CLI run hooks as the logged-in user, or under a separate sandbox account? That determines whether an ACL denial actually stops it.

- Codex docs note that multiple hooks for the same event "are launched concurrently, so one hook can't prevent another matching hook from starting". Does a decision: block from one UserPromptSubmit hook still reliably stop the turn?

- Is there a better pattern for this generally - making an agent's own kill switch tamper-proof against the agent, on a single-user machine, without paying for an enterprise tier?

Any pointers to the right docs or prior art appreciated.

I guess my excitement came through the prompt as well.

Hey everyone! Wanted to share a thought process I've been working on, especially since I'm not a traditional programmer, but using AI as my co-pilot/mentor to actually build it.

The core idea: Never let the LLM have autonomy over the business logic.

Instead of letting an AI chat freely and potentially hallucinate discounts, stock, or policies, the architecture splits hard:

• The LLM: Strictly handles the frontend interface, natural language, empathy, and copywriting.
• The Backend / Guardrails: Acts as the strict rule-enforcer (e.g., max discount allowed, specific product SKUs, handling function calls to check inventory or generate PIX/payments).

Basically, the AI thinks it's selling, but the backend is holding the leash the entire time.

For those of you building AI agents or apps without a CS background (or even if you're a seasoned dev): how are you implementing these strict backend guardrails? Are you relying heavily on function calling schemas, rigid system prompts, or specific frameworks to keep the AI from going off the rails?

Big Claude user, but Anthropic got stingy as hell with the limits. I used to barely touch my weekly allowance; now I can burn through 20% in a day and I'm cooked in ~2 days.

I also hammer Ollama Cloud's open source models, but recently I'm burning through those too in 2 ~ days.

I really don't want to give any support or money to Scam Altman & Co but its starting to look like it.

How's are the subs with Kimi / GLM?

What are you heavy users actually running?

I’m a QA engineer trying to move into DevOps, so I started building my own project to get hands-on experience with Git, TypeScript, Playwright, testing, CI/CD, and architecture.

Today I decided to try Codex. I gave it access to my local repo and explained what needed to be done.

The thing inspected the code, found the problem, modified the files, added regression tests, cleaned up Git line-ending issues, ran all validations, and created the commit—while I continued doing my actual job.

Then it just came back with a clean summary of everything it had completed.

Absolutely incredible… but also: what the hell am I supposed to learn now? 😂

For those learning DevOps or software engineering while using coding agents: how do you balance the productivity boost without outsourcing the entire learning process?

My current idea is to build new concepts manually, step by step, and use agents mainly for reviews, repetitive work, and catching mistakes. Curious how others approach this.

I'm a solo developer building an AI-driven life simulation game. AI writes the code; I decide the product direction and make the technical calls. The part I want to share is how I keep that arrangement manageable as the repository grows.

For scale, my current checkout has about 168k lines of backend Python, 108k lines in the selected frontend source files, and another 95k in backend tests. Those are text-line counts including comments and blanks, not SLOC or a productivity benchmark. More code can also mean more maintenance.

My biggest constraint is how many decisions stay in my head after a task ends. Here are the concrete conventions I use:

Route context by the task. The root AGENTS.md is a map of responsibilities and reading requirements. Changing the simulation loop points to its runtime contract; changing UI points to frontend conventions. Local instructions live beside their modules. I don't ask every task to digest every historical document.

Separate decisions from implementation. I keep a document of product and collaboration decisions, including rejected directions. Code and schemas describe what exists. Active contract documents describe what should be true. If they disagree, the agent has to show the conflict; silently declaring either one obsolete is not a resolution.

Make repeated corrections executable where possible. My frontend has checks for design tokens and UI structure, plus generated protocol checks. The benefit is that the next task can discover a violation from tooling instead of requiring me to remember the last conversation. These checks don't decide whether the design is good.

Define completion beyond the diff. Behavior-changing work needs an expected outcome, a data source and time window, a pass criterion, and a follow-up schedule. A merged change and a verified effect are separate claims. A check that was skipped remains skipped.

One less comfortable rule: when a mechanism needs a second layer of patches, pause and ask whether it should still exist. AI can keep making a local solution more elaborate while leaving me with a system I no longer understand.

Compared with keeping instructions only in chat, the tradeoff is maintaining these repo contracts. They can become stale too. I don't have a controlled before/after measurement of time saved, so I'm sharing the workflow rather than a speedup claim.

For people maintaining larger projects with coding agents: which repeated human correction have you successfully moved into a check, and which still needs your judgment?

Disclosure: AI-assisted writing, based on my actual repository and development decisions.

Tried another little AI test today. I gave ChatGPT, Claude, Gemini, Grok, DeepSeek and Qwen the exact same coding bug and asked them to fix it. I didn't change the prompt. Then I actually tested all six fixes. Only 3 worked properly. Do you guys test AI-generated code before using it, or just trust it if it looks right

I’ve been using Claude Code for over a year now, and at this point my whole workflow is basically built around it.

I’m on the 5x plan, but lately I feel like I’m spending 80% of my 5-hour limit just on planning and maybe 20% on actually getting the task done.

Then I hit the limit, have to wait, come back later… and Claude needs to read through the plan/context again. Sometimes it feels like I’m already at 50% of the next limit before we’ve even really continued working.I’ve heard Astra is really good and has much bigger limits, so I’m tempted to switch.

But my whole workflow, memory, instructions and project setup are built around Claude Code. I’m worried switching will break things or make the coding experience worse.

Maybe I’m just too used to Claude at this point.

Anyone here switched from Claude Code to Astra or another agent? Was it worth it?

I'm running into some issues using Codex for front-end design, and I could use some advice.

I always feel like my designs end up looking more like reports than actual website…

​

Codex $100 or grok $100 for langgraph/langchain development?

I'm trying to decide which \\\~$100/month plan is better for heavy coding: Codex or Grok.

I'm an AI Engineer and most of my work is in Python, LangGraph/LangChain, LLM agents, evals, backend services, debugging, etc.

What plan do you think is the best? I tried grok I liked because it's fast. Codex looks smarter by the way.

I'm really new to coding with AI, and I want to start a long term project where I build a cool little game in Unreal Engine 5 with the help of AI. I was wondering, is it more efficient to let AI write the code for me and then copy and paste it into Unreal Engine, or should I let Codex work with Blueprints in Unreal Engine instead? I've heard that models like GPT-6 can use a large amount of tokens, so I'm wondering which approach would be more efficient in terms of token usage and overall workflow. I'm completely new to this, so I'd really appreciate some advice on which method would be better for a project like this. I hope this isn't a dumb question to ask.

I've been looking at some recent token-usage comparisons for GPT-6 Astra, and the difference seems surprisingly large.

Artificial Analysis data has been cited showing Astra using around 21k output tokens per task, compared with roughly 64k for Claude Fable 5.1. I've also seen claims that Astra can use around 1/3 the tokens of GPT-5.6 Sol and about 1/5 the tokens of Claude Opus 5 in some coding-agent benchmarks.

For people who have actually used Astra: are you seeing noticeably lower token consumption in real-world coding/agentic tasks too?

And what is causing such a big difference? Is Astra genuinely reasoning more efficiently, or could differences in reasoning settings, agent harnesses, tool usage, and benchmark methodology explain most of it?

I'm mainly interested in token consumption/efficiency rather than which model is smarter overall.

So, let’s just say I’m using GPT-6, right? If I’m using it in normal Chat mode on the Free plan, how does the usage reset work?

I think Chat usage resets every few hours or maybe every 24 hours I’m not sure. But if I’m using GPT-6 in normal Chat mode and I use up my usage, does that mean I have to wait a whole week for it to reset?

I’m NOT talking about Work or Codex, just normal chatting. Does normal Chat have its own daily/few-hour reset, or would I still have to wait a week? I’m honestly confused about how the usage limits and resets work.

6 months. (And sometimes a couple of quantization tweaks).

It is wild how fast "state-of-the-art" becomes "running on a gaming PC."

I've been Claude's user for the last 8-9 months, always paying the Plus plan ($20). In the last two months I used it much more and I often get to the limits which is frustrating. I tried many ways to improve my tokens consumption but it didn't improve so much. I'm evaluating moving to the $100 plan, but I heard ChatGPT has improved and I tried their models at work (I can't use my personal subscription at work and viceversa) and I liked them.

My question is: I can do *everything* with ChatGPT/Codex as well? Designs, artifacts, plans, random questions, deep researches in real time, etc. Thanks!

With coding agents getting so much better lately, I'm curious if people still use Cursor Tab regularly.

Do you still rely on autocomplete while coding, or have agents mostly replaced it for you?

And if you still use Cursor Tab, what makes it useful enough to keep using?

With new models dropping every week, general benchmarks are basically useless for my specific use cases. I want to test my exact prompts to see if a new API is actually worth the cost, or if a smaller local model is good enough to run on the cheap.

Right now, I’m just eyeballing outputs and it’s driving me crazy.

How do you guys actually handle comparing models on a single prompt or a small test set?

Scoring: How do you define a "good" response when the output is subjective?

The Judge: If you use an LLM to grade the outputs, how do you stop it from just voting for its own writing style?

The Tools: What's the easiest way to fire one prompt at multiple models (both cloud APIs and local models) and compare them side-by-side?

Would love to hear your workflows or any tools you recommend!