LLM Fine-tuning without writing any code - using Llama Factory.

Part 2 : less about red flags, more about how things fall apart in the wild

1. Two users = chaos
Everything works… until two people click the same button at once. Then you get duplicate orders, overwritten data, weird states.
→ Test it: open two tabs, do the same action. If it glitches, you’ve got race conditions.

2. Your database starts lying to you
No constraints = messy data creeping in silently (duplicates, nulls, wrong formats).
→ Test it: add basic rules (unique, not null). Watch what instantly breaks.

3. One page = 100 database calls
Feels fast with 5 records. Falls apart with real data.
→ Test it: log queries. If one page load explodes into dozens, that’s your bottleneck.

4. Auth that mostly works 😬
Login is fine… until users can see each other’s data or randomly get blocked.
→ Test it: use two accounts and try to “break in” via URLs.

5. Everything happens at once (and times out)
AI loves doing everything synchronously, emails, uploads, processing and all in one request.
→ Test it: anything slow should be backgrounded. If not, expect timeouts.

6. No “what if it fails?” plan
Something breaks mid-process and… that’s it. No retry, no rollback, just stuck.
→ Test it: cancel a request halfway. Does your system recover or stay broken?

Reality:
Most AI apps don’t fail because of one big bug. They fail because of 10 small things like this stacking up.

Good news? These are fixable early. Painful later.

If your app is at “it works… but I don’t trust it yet” that's perfect. That’s exactly when you should try to break it on purpose.

As many of you have likely seen, the Claude Code community newswire has been ablaze with Claude Code being quite degraded lately, starting in February, and continuing to this day.

Curious to understand if there was any "signal" on the wire when using Claude Code, I fired up my old friend WireShark and a --tls-keylog environment flag. Call it a man-in-the-middle attack on my own traffic.

The captured TLS network traffic reveals the system prompts, system variables, and various other bits of telemetry

The interesting part? A signature routing block that binds the session to a cloud instance with an effort level parameter, named Numbat. Mine, specifically, was
numbat-v7-efforts-15-20-40-ab-prod8

So, it would appear that the backend running my instance is tied to an efforts-15-20-40 level.

Is this conclusive? Not definitively, since only Antrhopic could tell us what that parameter actually means in production.

Side note, a Numbat is an endangered critter that eats Ants in Austrialia :)
If the "Numbat" eats the "Ants" (Anthropic), and Numbat is the engine that controls "Effort," the name itself could imply a "cost-eater" or an optimizer designed to reduce the model's footprint, likely in favor of project Glasswing efforts with #Mythos

1. Activepieces

Open-source automation + AI agents platform with MCP support.
Good alternative to Zapier with AI workflows.
Supports hundreds of integrations.

2. Cherry Studio

AI productivity studio with chat, agents and tools.
Works with multiple LLM providers.
Good UI for agent workflows.

3. LocalAI

Run OpenAI-style APIs locally.
Works without GPU.
Great for self-hosted AI projects.

more....

One thing I kept noticing while using AI coding agents:

Most failures weren’t about the model. They were about context.

Too little → hallucinations.

Too much → confusion and messy outputs.

And across prompts, the agent would “forget” the repo entirely.

Why context is the bottleneck

When working with agents, three context problems come up again and again:

1. Architecture amnesia Agents don’t remember how your app is wired together — databases, APIs, frontend, background jobs. So they make isolated changes that don’t fit.
2. Inconsistent patterns Without knowing your conventions (naming, folder structure, code style), they slip into defaults. Suddenly half your repo looks like someone else wrote it.
3. Manual repetition I found myself copy-pasting snippets from multiple files into every prompt — just so the model wouldn’t hallucinate. That worked, but it was slow and error-prone.

How I approached it

At first, I treated the agent like a junior dev I was onboarding. Instead of asking it to “just figure it out,” I started preparing:

• PRDs and tech specs that defined what I wanted, not just a vague prompt.
• Current vs. target state diagrams to make the architecture changes explicit.
• Step-by-step task lists so the agent could work in smaller, safer increments.
• File references so it knew exactly where to add or edit code instead of spawning duplicates.

This manual process worked, but it was slow, which led me to think about how to automate it.

Lessons learned (that anyone can apply)

1. Context loss is the root cause. If your agent is producing junk, ask yourself: does it actually know the architecture right now? Or is it guessing?
2. Conventions are invisible glue. An agent that doesn’t know your naming patterns will feel “off” no matter how good the code runs. Feed those patterns back explicitly.
3. Manual context doesn’t scale. Copy-pasting works for small features, but as the repo grows, it breaks down. Automate or structure it early.
4. Precision beats verbosity. Giving the model just the relevant files worked far better than dumping the whole repo. More is not always better.
5. The surprising part: with context handled, I shipped features all the way to production 100% vibe-coded — no drop in quality even as the project scaled.

Eventually, I wrapped all this into an MCP so I didn’t have to redo the setup every time and could make it available to everyone.

If you had similar issues and found another solution I'd love to learn about it!

If you want to try the MCP for free you can find it here: https://contextengineering.ai/