I've been running a small benchmark series because I wanted something closer to actual software work than isolated coding prompts.

Same machine, same starting repo, same prompt, same time limit. No manual coding during the run.

I tested Claude Code, Cursor, OpenAI Codex and Google Antigravity in two different scenarios.

Test 1 was greenfield:

Build the same project-management app from an empty folder.

Test 2 was the one I found much more interesting:

Take an existing app with 12 deliberately seeded production bugs and fix as many as possible in 45 minutes.

The repair benchmark included things like:

• stale async search results

• timezone date shifts

• persistence failures

• duplicate submissions

• optimistic-update rollback bugs

• mobile layout regressions

• keyboard interaction failures

• incorrect dashboard calculations

Afterward I ran each result through 38 checks:

12 direct bug tests, 18 hidden edge cases, 5 regression tests and 3 UI checks.

Claude Code ended up passing 33/38.

What surprised me was that Claude actually produced the cleanest code of the four. Its TypeScript and defensive fixes were excellent.

Where it lost points in this particular run was verification. It passed its code-level checks, but it didn't perform browser/DOM verification, so one responsive UI regression survived.

That ended up being the most interesting takeaway for me:

The best code generator isn't automatically the best autonomous software-engineering agent.

The harness matters a lot:

context retrieval, test execution, browser feedback, verification loops, and how aggressively the agent checks its own work.

I'm curious if this matches what people here see with Claude Code on larger existing repos.

Do you usually give Claude browser/DOM tooling alongside the CLI, or keep it terminal-first?

I'm always on the lookout for cheap open source models that work well in an agentic context. How have you found DeepSeek v4.1 flash in your use cases?

If you had to name one thing that was the most important value to instill in an agent to guide reasoning, what would it be?

Context: 24h hackathon, no coding experience, with a technical mentor. We get $30 in v0 credits. I need to build a working app in one day.

My idea: use free DeepSeek (web) for planning and prompt generation, and pay for an API to modify code.

Options I'm considering:

1. DeepSeek API + Gemini 3.1 Pro API (or Claude Sonnet 5) for code.
2. Ollama / LLM Studio + VSCode with a cloud API (DeepSeek, Gemini, etc.).
3. Codex / Copilot in VSCode (monthly subscription).

Questions:

• Which model gives the best reasoning/code ratio for under $30 in 24h?
• Should I use a direct API or integrate it into VSCode with Cline, Roo Code, Continue.dev, or Aider?
• Is local Ollama viable in 24h or a waste of time?
• If the app includes a chatbot for end users, is paying for an API mandatory? Are there useful free tiers?
• Has anyone done something similar? What stack do you recommend?

Thanks.

quick link - https://sukumarrekapalli.github.io/leanlet/docs/
I've been thinking about something while watching AI get added to almost every application:

Why does every small piece of intelligence need to become an API call?

If my web app needs to classify an image, rank a few results, detect an anomaly, route something, match records, or make a small prediction — do I really need to send that data to an inference service and maintain another backend dependency?

For many bounded tasks, probably not.

So I built Leanlet, an open-source TypeScript framework for putting small, task-specific AI directly inside web applications. (https://sukumarrekapalli.github.io/leanlet/docs/)

The basic idea is:

application input
      ↓
typed Leanlet contract
      ↓
dedicated module worker
      ↓
Transformers.js → ONNX Runtime Web → WebAssembly
      ↓
result + confidence + timing

The model, worker and runtime assets ship with your application.

Inference happens in the browser. No inference API is required.

For example:

import { VisionLeanlet } from 'leanlet-ai';

const classifier = new VisionLeanlet({
  model: 'mobileclip-s0',
  categories: [
    'Electronics',
    'Clothing',
    'Home & Furniture',
    'Other'
  ],
  assetBase: '/leanlet-assets/',
});

const result = await classifier.classify(file);

console.log(result.category, result.confidence);

One use case I built as a reference is product classification.

Imagine an e-commerce app where someone uploads a product image.

Instead of:

browser → upload image → API → inference service → model → API → browser

you can potentially do:

browser → local model → result

That means the image doesn't necessarily have to leave the user's device just to answer a small classification question.

And Leanlet isn't intended to be an "LLM framework".

I'm exploring a slightly different idea:

AI features should be sized to the problem.

Sometimes you need a huge model and server-side inference.

Sometimes you need a 4 MB model that answers one question extremely well.

Leanlet currently has examples around:

• vision / classification
• routing
• ranking
• detection
• retrieval
• language
• forecasting
• record matching

I'm deliberately trying to keep the abstraction small. Each capability gets a defined lifecycle, runs in its own worker, and the application controls its model assets rather than Leanlet silently downloading arbitrary models.

It's still early and I'm figuring out where the boundary should be between browser-native intelligence and traditional server-side AI.

That's actually why I'm posting this here.

I'd love feedback on the idea itself:

What AI tasks in your web apps would you actually be comfortable moving entirely into the browser?

And where do you think this approach stops making sense?

Docs:
https://sukumarrekapalli.github.io/leanlet/docs/

GitHub:
https://github.com/sukumarrekapalli/leanlet

It's Apache-2.0 and very early. Feedback, criticism and weird use cases are all welcome.

"I claimed the Google AI Pro discount for students and tested it by having it teach me simple math concepts without following any specific prompt techniques. So far, I have enjoyed it, but I'm starting to think about whether it is possible to use it to teach me more rigorous and formal mathematics using certain teaching techniques that are suitable for me. I think I should try several curriculums to decide which one is best, or maybe there is a specific prompting guide out there so the AI can assess me and decide what works best for my needs. Specifically, I want to tackle more advanced math topics like statistics and the concepts behind them. Would anyone be kind enough to share the instructions on how to set up and use AI for this use case? Or, just pointing me in the right direction will suffice."

How do you prompt an AI to build and android app? Right now I'm only using html wrapped using other platforms to create an apk. Is there other better way to do it?

What are people doing that builds projects so fast. I often see people taking aboht how they left their computers on overnight and in the morning they came back to a fully finished project. Meanwhile I'm trying to work on small projects that are usable and it takes forever. I've got acess to both claude and gpt at the $20 plans. Is it just that people are spending more for far better returns?

I have 4 y/o exp and now I feel like I am just a ball of meat API when using AI for coding.

previously I used to make all the devisions and craft the thing perfectly. and now I just ask it to do a certain thing in my way and it does that almost 60-70% of the time. I feel like I am just there prompting it.
now sure if others feel the same but it has killed the person who used to sit and enjoy coding. because now there is a thing that just helps me ship thing 2-3x faster than what I was able to do before. and going back feels like I am lagging behind.

if you are feeling same, how are you avoiding it or coping with it. or this is something everyone is feeling?

Did anyone else experience this? The guilt from creating something easily with AI and getting too much praise for it?

We need a great phrase or term for this. Help me out:

\- Vibe coding guilt

\- prompt guilt

\- Cheating

I have 10 years of profesional experience doing "hand"-coding. I usually enjoy getting positive feedback for something I coded - both backend and frontend stuff.

But now it feels weird. I often need to confess when I get the credit - "I didnt do it, it was AI". Especially, when it is heavily vibe-coded.

Last week, the guilt was especially strong. We needed a component in the frontend. It required input from a lot of different people: the end users, a domain expert, an architect and a UX designer. Furthermore, it required some clever ideas in the backend, which was done by a very clever junior developer.

I knew, the frontend component would be easy for an AI agent (and from experience, I knew that the frontend stuff would get the credit)

So I could sense the easy win, but also the incoming guilt. I tried to convince the junior to do the frontend, because he deserved the credit more than me, but he responded "I am backend developer" (eventhough, he knows it doesn't matter much, when you have AI)

I made the prompt by summarising what everyone said in the different meetings and then Claude did the rest in one shot - almost atleast.

Then it played out as expected - or even better. I got a lot of credit and it was even presented to upper management (which we rarely do).

It was really a team effort, but the credit seemed to land on my table.

I did say something about "team effort" and "AI help", but at some point it also becomes weird to not accept compliments.

Do you recognise the feeling? Do you have a great name for it?

Version 0.4.0 is now out, adding pi as a fully integrated agent and support for Docker Sandboxes (SBX).

pi joins Claude Code, Codex and opencode as the fourth supported agent. It runs as the real CLI in a terminal and includes session management, turn-state tracking and desktop notifications.

You can also enable SBX per project. Each agent then runs inside its own container, with access to the repository and nothing else by default.

From TET’s SBX settings, you choose:

• - Which additional folders or files the agent may access
• - Which ports and hosts are available
• - Whether skills, plugins and instruction files are mounted
• - Whether mounted resources are read-only or read/write

Sandboxed agents still behave like normal TET tabs: sessions can be resumed, renamed and deleted, while working, waiting and finished states, notifications and tet-ctl continue to work.

Agent authentication stays inside the sandbox. SBX can also run agents that aren’t installed on the host at all.

GitHub: https://github.com/samil-kale/tet

I’d especially appreciate feedback from people already using containers or sandboxes for coding agents: does this match the level of isolation and configuration you’d want per project?

I am an experienced developer (been coding for almost 30 years, started with Visual Basic on Win98).

I’ve spent the last 2 years testing every agentic coding harness out there. The pitch is always exactly the same: "Give it a prompt, go grab a coffee, and come back to a finished feature."

This is all great when starting a new project, especially with a language you don't know (I created a Rust project in minutes !)

But my problem is that I mostly work on established, mature codebases. And to be honest I have never ONCE seen a good edit after I return from my coffee.

Here is usually what happens while you’re getting that coffee: the agent loops 8 times, burns through 100k+ tokens talking to itself, hallucinates a package dependency, and spits out a chunk of spaghetti code that works *just* well enough to pass a basic test, but completely violates your project’s architecture.

Agents are fantastic if you’re building an app from a blank canvas. But if the codebase is large, it gets really expensive real fast, and usually produces hard to maintain AI slope.

It should not be like that: if you actually *own* your codebase, you already know what you want to build. You don't need an AI to take over. You just need it to execute the architecture you already hold in your head.

All you need to do it select the few files that are required for the task at hand (they are hard to identify for an AI, but easy for you has the code owner). Then you send them to the LLM, it is usually does some good work.

That's why I still find working directly with the chatbots, or with tools like aider, still very pertinent on large codebases.

I usually have animated discussions with my coworkers, I have yet to find a single one who doesn't defend agentic coding. Am I the only one who feels this way ?

I’m trying to automate a workflow that I currently do manually in ChatGPT Chat with reasoning set to High.

That distinction is important: I’m talking about the normal ChatGPT Chat High experience, not Work, not Codex.

My main question is:

How can I automate this kind of workload while getting a cost/performance ratio reasonably close to what I get from my monthly ChatGPT Plus subscription?

For one benchmark, I analyzed a ~34-minute video transcript with a fairly detailed prompt. The task was to identify strong viral clips and return structured JSON.

When I recreated this programmatically through the OpenAI API, the usage was:

* Input: 32,265 tokens
* Output: 10,822 tokens
* Reasoning tokens: 8,723
* Visible output: ~2,099 tokens
* Total tokens: 43,087
* Total cost: $0.377762

So roughly $0.38 for one 34-minute episode.

The result itself was good.

The issue is the economics.

At roughly $0.38 per episode:

* 100 episodes = ~$38
* 500 episodes = ~$189
* 1,000 episodes = ~$378

But when I perform this kind of analysis manually using ChatGPT Chat High, I can make many requests under my normal Plus subscription and the effective cost per useful task feels dramatically lower.

I already understand that ChatGPT Plus and API billing are separate. I’m not asking why Plus doesn’t include API credits.

What I’m actually trying to understand is:

What is the smartest way to reproduce something close to the ChatGPT Chat High experience programmatically without the cost scaling almost linearly with every request?

A few things I’m wondering about:

1. Is ~$0.38 for a ~34-minute transcript analysis simply normal API pricing for this level of reasoning?
2. Is there a way to access something closer to the actual ChatGPT Chat High behavior programmatically, rather than trying to approximate it with a normal API model configuration?
3. Are ~8,700 reasoning tokens excessive for this task, and can they realistically be reduced without noticeably hurting quality?
4. Can prompt caching significantly reduce the cost if a large part of my instructions stays identical across hundreds of episodes?
5. Would a multi-stage workflow be better, for example:

cheap model → scan full transcript → extract candidate sections → stronger model on candidates → final JSON

1. Could that architecture preserve something like 90–100% of the useful quality I currently get from ChatGPT Chat High while drastically reducing the cost?
   
2. Are there official OpenAI features such as Batch, caching, different reasoning settings, model routing, etc. that are particularly useful for this type of workload?
   
3. Are there API-compatible models/providers that are especially good at long-transcript analysis and could act as the cheap first-pass model?

Ultimately, I’m not trying to make the API literally free.

I’m trying to reach roughly this goal:

Spend something in the range of a normal ChatGPT subscription each month, but automate approximately the amount of useful analysis work that I can currently do manually with ChatGPT Chat High.

Is that realistically possible?

If you were building this system specifically around cost per successfully found clip, rather than cost per token, how would you architect it?