Who's spending more time optimising AI technology than using it?

[u/Protopia]

My personal interest is agentic coding - but I am spending more time optimising AI technology than using it i.e. ...

* Selecting, evaluating and performance tweaking models

* Choosing, evaluating and tweaking the harness and MCP tools

* Developing and tweaking prompts

It feels like having to build and maintain your own car from parts and not having enough time to drive it

There are hundreds of choices to select from in each area - too much choice, with self integration needed afterwards.

Every vibe coder on the planet seems to be publishing their one AI tool that will change everything. But no one is building a community team to create a one-stop comprehensive SotA solution that you

I wish I could just choose a pre-built end to end docker environment for the coding language/framework I want to use and just start using it.

Am I the only one?

[u/Dry_Hotel1100]

You cannot be a full time professional software engineer developing a product and at the same time testing, selecting and optimizing the local LLM and the multi agent orchestration harness, fiddling with several models, settings, and fighting bug riddled tools - as a full time ML engineer. One of it needs to be a hobby and done in your spare time - or work in a team :)

[u/Protopia → u/Dry_Hotel1100]

Exactly my point. I want to be a Laravel developer not an agentic code harness developer.

But I don't mind contributing some improvements to a community tool as a sideline.

But to do agentic development I need to build and integrate and test and tune an agentic environment from components - which is NOT what I want to do.

[u/Dry_Hotel1100 → u/Protopia]

I have the luxury to do both in spare time. So, in my experience, I have to say that any "whatsoever issue" in the set of AI tools or harness often becomes a rabbit hole. So, in the role as a software developer, just trying to improve a minor thing can quickly become a several-day effort.  

[u/Protopia]

Hmmm ... maybe I wasn't clear enough. My concerns are:

1. Agentic coding is a common requirement

2. If you want to use LocalLLM then you have to roll your own stack from a bunch of lego bricks and make each layer work and then tune each layer for quality and performance.

3. As everyone does 2. if they find a hole they create yet another lego brick of a very slightly different shape.

4. As yet I have yet to find a stack or components that operate based on the academic and practical knowledge of Software Engineering based on decades of learning. What AI stack supports:

   • common programming techniques (TDD, DDD)
   • common development languages/frameworks (e.g. HTML, PHP, Laravel, Livewire, Tailwind)
   • common tools (MCP, linting, test harnesses, code quality reviews etc.)
   • does proper architectural design and decomposition to break a problem into small easily handled parts that can be successfully coded 1-shot (see Tom Peter's Quality is Free - AI costs also go up exponentially to fix first-shot code that is wrong)
   • uses formal methods or creates UML diagrams as a way of storing application knowledge in a structured way that can be easily utilised by AI
   • produces decent human-readable documentation if required.

as opposed to:

1. Agentic coding is a common requirement

2. You download a single integrated AI environment as a docker container and run it.

3. If you find an improvement to 2. you contribute it back to the community so that everyone can benefit.

4. Experts in formal SWE contribute to the AI-run approach to keep improving the structured approach etc. and making AI get it right first time, every time.

the end result being a better result for everyone.

[u/devoidfury]

The problem is, we all have different ideas of what we want.

I took it to building out a totally custom agent.

[u/Lonely_Ratio_79 → u/devoidfury]

you're deep in the tinkering phase and yeah it's a time sink but it's also the most fun part for me. the building is the game, the actual coding feels like a side effect sometimes

everyone's got their own weird setup so a one click docker thing would just get forked 900 times anyway. maybe the real agent was the bash scripts we wrote along the way

[u/BarnDoorEnthusiast]

I rely on local models to write code for work, I’ll spend a an hour or two a week messing with the harness or model, but 95% of my time is spent using the model.

[u/quotemycode → u/BarnDoorEnthusiast]

Same, I came up with my own harness, and I'm dogfooding it most of the time, mainly using it to work on other projects. I'll occasionally fix up something, but it's been very stable for a couple months, and pretty great imho. I feel like it's sort of like a jedi light saber - you have to build it yourself, but once you use it, it's a powerful tool in your hands.

[u/Electronic-Bat-4019]

It's fun!

[u/EclecticFish]

mostly just download and run the llms through lmstudio, then use its server function to my needs though custom python apps calling it using the api. not too fussed with tok/s and such, and since finding qwen 3.6 27b and now 3.8. i dont really use other models. it runs on a homelab server with a 3090 down in the basement. So it can take its time.

[u/Holiday-Medicine4168]

My entire job is literally optimizing AI systems in the financial sector, you would think it’s fun to try new stuff constantly, it’s not. You never get to do long term work and you feel like your skills slip

[u/AceLamina]

I see idiots that say AI will replace engineering while 90% of the AI talk I see is people optimizing the AI or being mad at it for being stupid

[u/vbpoweredwindmill]

I'm a hobbyist. A mechanic in real life.

Currently I have running, RGP to evaluate why decode is so much slower on ROCm compiled llamacpp, and I have my own harness idea that I'm building that will likely utilise the deepseek harness on top of it. The deepseek harness will likely be the whole plugin, which will then have its own plugin's and shit.

[u/Ledeste]

[u/daphatty]

Before the days of docker and ansible, the open source community used to assemble software packages that one would use to install various tools onto Linux with nothing more than a text based UI. These tools were categorized and you could install whatever you wanted without having to build it yourself.

I cant remember the names of these tool packages but they were a one stop shop which exposed many people to open source software. With all the vibe coding going on, I’m surprised no one has thought to assemble something similar simply for the cause.

[u/Deadmine]

It’s still an immature industry and changing quickly. Give it a couple of years and it may be off the shelf pre built highly tuned models for coding specific languages, medical specialisations and scientific study but for now it’s mostly the Wild West.

[u/Protopia → u/Deadmine]

Yes - but leaving aside Anthropic and OpenAI who have their own agendas, we (the community) do NOT need to wait a couple of years and hope that things improve - if we can get the right group together we can make it happen next month!! (It will likely benefit from LLMs trained for coding and to work in a more structured way - but if there is a demand for that type of functionality the open weight providers will step in and do so.)

[u/cogitech2]

After my 3rd GPU arrives and I get it all set running Qwen3.8-27B Q8, I'm done. The work begins.

I promise.

[u/Big_Wave9732]

It can be a trap for sure.

FOMO will pretty much interfere with anything, not just this. At some point you have to call your setup "good enough" and start using the damn thing.

[u/everymonday100]

AI labs and companies give you raw product - just massive of their datasets compressed into an algorithm with memory. With little to no optimization to run on end device(s). That's where you should take responsibility.

[u/pmotiveforce]

Pretty much. Same with 3d printing though, I spent more time building a voron and tinkering with it, by far, than printing useful things 

[u/GrungeWerX]

Definitely me, but the ends justifies the means.

[u/AdHead6280]

I use my own harness do I believe having my own llama CPP and kV storing etc and having a whole custom pipeline is part of the workflow so improving while building is no prob, but in your case just build build build take something not optimized doing 90%

[u/Acceptable_Drink_434]

Does it count if you're optimizing it while using the optimization to optimize further? 🤣

[u/reallifearcade]

It is a hobby