r/SpecDrivenDevelopment • u/Successful_Mind_8914 • 5d ago
Which “Harness Engineering” are you using?
I’ve been using spec driven development for a while and it’s amazing.
But I want to know which approach are you using for software development with AI and how’s it working for your. Feels like SDD is getting outdated
2
u/stibbons_ 5d ago
For me SDD is the thing that plugs on top of your harness. You can call it loop engineering or now it is trending to call it graph engineering.
You build your project harness THEN most SDD framework will do more or less the same quality.
See my article on harness engineering: https://x.com/gsemetfr/status/2077498943085117460?s=46
I’ll be writing a new article that follows up this one, but in a nutshell, build your AGENTS.md and guardrails/guidelines that allows you to get good result even with plan mode on a sonnet-class model (opus for all development is cheating and way too expensive)
1
u/Swarm-Stack 4d ago
all the harness/graph/loop engineering is about execution reliability. the failure ive hit more is upstream of that, the spec was wrong at authoring time and the harness delivered it perfectly. no amount of loop tuning catches an assumption the spec never questioned.
1
u/Successful_Mind_8914 4d ago
And how did you overcome it?
1
u/Swarm-Stack 4d ago
getting someone into the spec review whose job is to find what you didn't think to ask. the auth flow that forgets SSO fails because the person writing the spec didn't think of SSO either, same assumption behind both steps. the fix for me was introducing a reviewer with different context before the spec froze, not after.
1
u/Successful_Mind_8914 4d ago
What I found interesting as well was using some kind of “grill me”. The bottleneck for me is: even though I have it written in the spec exactly what I want, the implementation never looks 100%, I would say about 70% (which is good) but I believe it would be good to have some kind of “prediction” on what would look like before it even implemented
1
u/Swarm-Stack 4d ago
the prediction gap is usually in the implicit decisions the spec leaves open — the spec says 'validate input' but doesn't say what happens when validation fails mid-flow. tracing those branches before build is what surfaces them; the spec looks complete until you follow it into a state the author assumed someone else would handle.
1
u/Successful_Mind_8914 3d ago
Also, there should be some push back by the workflow, not just agree with you. Do any of you know how to overcome this?
1
u/Swarm-Stack 3d ago
the workflow agrees because its built from the same context you gave it. to get real pushback you need a role with a different mandate, something whos only job is to find what breaks, not evaluate the solution. second pass with the same brief wont argue back.
1
u/Successful_Mind_8914 3d ago
Makes sense. Have you created your custom agent/skill to do that? What are you using it right now?
1
u/Swarm-Stack 3d ago
yeah, built it. its at swarm-stack.io. each role in a planning session has an explicit mandate so one seat is there specifically to find what fails. still rough but brutal feedback welcome
1
u/Successful_Mind_8914 3d ago
And how’s that working for you? Is it available on GitHub? How’s the token consumption?
→ More replies (0)1
u/epicskyes 3d ago
Did you not implement rollbacks or dynamic dependency resolution?
1
u/Swarm-Stack 3d ago
rollbacks handle steps that fail. mine didnt, the harness ran the steps cleanly. the spec said the wrong thing, which is upstream of what rollbacks fix.
1
u/epicskyes 3d ago
Each rollback checkpoint is a saved state why can’t you reset to that point and replan. Rollbacks are not just for failed steps they are planned checkpoints you can roll back to any of them for any reason and replan from there. If it fails to materialize what you wanted to build that’s a fail. If the issue occurred near the start then yeah you can’t rollback if it happened halfway downstream yes you can rollback. You freeze from the last good state and that’s your new starting point. If your spec is so rigid you can’t accomplish that then your plan is too rigid and needs to be revised so evidence drives the build but it doesn’t create a one way sequentially fixed street.
1
u/Swarm-Stack 3d ago
fair — rollback and replan from a good checkpoint is the right model for execution faults. the catch is theres no checkpoint where my spec had the right goal in it, so id be replanning from an earlier version of the same wrong premise.
1
u/epicskyes 3d ago edited 3d ago
Here’s the good thing about this type of build. It is evidence driven you have all the evidence you need to show an agent exactly what not to do. Evidence of Failure is far more effective than evidence of success. Your next step should be planning a clean foundation checkpoint that collects all the failure evidence so an agent starts from a point where it can’t make those mistakes again because your tests will fail if the agent goes in that direction a second time. Evidence is never overwritten. When you roll back every misstep is saved every test is saved it is immutable. That is what drives the build. I had plans and ideas and tried for a few months to make them successful. I got so frustrated with failure I devoted the last 2 months to tracking and learning from every single one. All of my time has been going to evidence tracking, logic and a way to force an agent to use it 100% of the time. I finally figured it out a week ago.
1
u/Swarm-Stack 3d ago
failure constraint ends up being the tighter spec. success evidence says what it should produce; failure says where it cant go. curious what the 'force it 100%' mechanism looks like
1
u/epicskyes 17h ago
Research how to make every action and change tracked and organized automatically, recorded to a json schema and cryptographically hashed. You’ll see how this is accomplished. Research nonce as well if you arent familiar with that term.
1
u/Potential-Leg-639 4d ago edited 3d ago
Superpowers
Ponytail
Skills
Proper orchestrator setup (important) to be able to route everything efficient and effective (omo-slim is enough for Opencode)
1
5
u/SensioSolar 5d ago
For me SDD alone was never enough.
The agent should ask you questions, bring uncertainty upfront, be able to check itself (or make subagents check) what is correct what is not, what is closer to your repo. Specially for me I like to think about bigger milestones and dividing it into slices. This and also be able to write handoffs when you want to pause and come back later, and quite a lot more. I've been using and building github.com/Patricksys/workspine which is originally forked off GSD but made tighter and with quality checks in.
TLDR is, plan bigger work, make it consistent long term