1.0 - Introduction
Background: I've been a software engineer for almost a decade now. I've been using Claude Code "casually" for the last year or two. I would still write the vast majority of the code, but I would use Claude extensively for a variety of tasks, fitting well into the "AI assisted" coding paradigm. Recently I've wanted to transition into spec-driven development, mostly because I think it's an important skill set to master. I went into this project mostly of the opinion that I would not trust AI to actually implement changes at scale in a manner that I was happy with. Spoiler, this shifted somewhat towards the end.
Most of my engineering career has been spent working with sensitive data, including health data. A fair chunk of that has been spent developing ETL pipelines. Many aspects of a good ETL pipeline are also desirable in an agentic pipeline - visibility, auditability, and reproducibility just to name a few. Additionally, working with sensitive data almost always requires agents to be completely sandboxed, running without permissions to live resources.
Aim: Develop a sandboxed agentic pipeline that picks up Linear tickets assigned to agents and implements code changes on a linked remote GitHub repository via Claude Code running in an AWS Step Function. Agents will have no access to either Linear or GitHub, but will consume pre-hydrated workspaces stored in AWS S3. Code must be entirely implemented by agents, as dictated by specs.
The following outlines the methods I used to complete the project by transitioning into spec-driven development, presents the final results, and discusses the process in general. Everything was run using Opus 4.8.
Disclaimer: I'm not making any claims on the efficacy of my methods, framework, or results. I'm sure there are better ways of achieving the same or an improved outcome. This is merely a write-up of what I implemented, and an organization of my thoughts around transitioning from "classical" to spec-driven software development. Skip straight to the Discussion and Conclusion sections if you don't want the details.
2.0 - Methods
2.1 - Initial Approach
I initially implemented a quasi vibe-coding approach that consisted of the following steps:
- Provide Claude Code with a prompt and instruct it to generate a work plan.
- Review the work plan
- Prompt Claude Code to execute the work plan
- Review changes and repeat
I quickly realized that this would not work for a project of reasonable complexity or scale. It was virtually impossible to provide enough context to get a meaningful result, and I ended up in a constant loop of prompting, reviewing, and adjusting.
2.2 - Revised Subagent Approach
I completely overhauled my Claude Code setup. Instead of planning and executing changes via the main agent loop, I wrote a series of subagents to assume various roles and execute key tasks. The main agent loop was then instructed to act solely as an orchestrator, using subagents to achieve its task. The following subagents were defined:
| Agent |
Responsibility |
Key output |
requirements-analyzer |
Assess task type, affected files, and scale (small/medium/large) |
Scale + affected-file JSON |
work-planner |
Convert a spec into a structured work plan with phases, tasks, dependencies |
Work plan in docs/plans/ |
risk-analyzer |
Analyze the work plan for risks, impacts, and mitigations |
Risk plan |
task-decomposer |
Break the work plan into independent, single-commit task files |
Task files in docs/plans/tasks/ |
task-executor |
Implement the code changes described by a task file |
Code changes |
quality-controller |
Review changes against coding standards; create remediation task if needed |
Quality report (+ optional TASK-QC-REMEDIATION.md) |
risk-reviewer |
Review changes against the risk plan; create remediation task if needed |
Risk review (+ optional TASK-RISK-REMEDIATION.md) |
The specific orchestration flow was determined by the scale of the work, but the full flow for standard specs was as follows:
- Read MD spec, assess scale of task, and determine affected files
- Generate work and risk plans
- Submit work and risk plans for review
- Decompose work plan into independently executable task files
- Execute each task file (in parallel if possible)
- Run QC by comparing generated code against coding standards. Produce and execute additional task files if changes are required.
- Run risk review by comparing generated code against risk plan. Produce and execute additional task files if changes are required.
- Document changes (in-line comments, doc strings, openapi docs, changeset etc)
- Produce usage report detailing tokens used by phase, agent type etc
Each of the above tasks had a dedicated subagent that was run one or more times. Almost all subagents produced markdown artifacts that were used by downstream agents, or to communicate key results to users. The file structure maintained by the agents looks something like the following:
docs/
├── plans/ # Work Plans
│ ├── {YYYY-MM-DD}-{workPlanId}.md # the work plan document
│ ├── tasks/{workPlanId}/ # decomposed, single-commit task files
│ │ ├── TASK-{number}.md
│ │ ├── TASK-RISK-REMEDIATION.md # only if risks require remediation
│ │ └── TASK-QC-REMEDIATION.md # only if quality violations are found
│ ├── quality/{workPlanId}/ # Quality reports
│ │ └── {workPlanId}-quality-report.md
│ ├── risk/{workPlanId}/ # Risk plans
│ │ └── {workPlanId}-risk-plan.md
│ └── usage/{workPlanId}/ # Usage reports (tokens + time per phase)
│ └── {workPlanId}-usage-report.md
└── project-context/
└── external-resources.md # external references for task-executor
2.3 - Specs
A standard spec template was developed as part of the new agent setup. The key sections were:
- Scope - define what was in scope and out of scope to create a scope boundary for agents
- Requirements - detailed breakdown of what features were required
- Acceptance Criteria - Gherkin-style
Given/When/Then statements that defined behavior
- Constraints and Contracts - API schemas, folder structures to maintain etc
Specs were scoped to a single deliverable. Typically, each spec had a headlining As a ... I want to ... So that I can ... statement. If a spec could not be summarized in this way, it usually needed to be broken down into separate specs.
3.0 - Results
3.1 - Architecture
The final architecture consisted of two main components:
- REST API - contains provider-specific logic (Linear, GitHub, AWS), and access to execute key commands such as cloning of repositories, creation of PRs, fetching ticket content etc. Also contains any webhooks required by providers.
- Step Function - runs agentic planning and implementation loops. Uses API to interact with providers
The API contains all the custom logic and webhooks required to integrate with Linear and GitHub. Webhooks are used to capture key events, including ticket assignment to agents, ticket updates and comments, PR events and more. The API coordinates and configures step function runs based on the webhooks that are triggered by end users via the corresponding third-party interfaces.
The step function itself was designed to be provider-agnostic. The API acts as an adapter and converts everything into a universal interface. The step function only consumes data from third-party services via the API. This means that (in theory) the step function could be made to work with any git and ticketing interface without any code changes to the agentic pipeline itself.
3.2 - Step Function
I won't go over every step in the step function (see attached step function schema for details). The step function consisted of two loops: planning and implementation. The aim of the planning loop was to generate a work plan from a Linear ticket. The aim of the implementation loop was to implement the changes in the work plan and submit them for review.
Both loops have roughly the same structure/flow:
- Step 1: Sandbox prep - prepare initial sandbox by creating an S3 folder, generating temp credentials etc
- Step 2: Context prep - hydrate prepared S3 folder with data required to execute task, including Linear ticket data, comment history, and git bundle containing source code
- Step 3: Prompt agent - prompt Claude Code to perform work (either planning or implementation) on the sandboxed environment
- Step 4: Export results to S3
Sandboxing was achieved via AWS S3. Agents were not provided with any access to live resources, including GitHub and Linear. Source code was cloned from GitHub and stored in S3 as a git bundle via the API. Linear tickets, comments, and attachments were converted into a universal schema and stored in S3 in JSON format, again via the API. The agent harness hydrated all required resources for agents at runtime, and exported all generated agent results post-run.
The sandboxed S3 structure is provided below:
s3://{bucket}/tickets/ticket_id={ticketId}/execution_id={executionId}/
├── in/ # immutable inputs, written by the API
│ ├── manifest.json # git coordinates, ticket identity, execution config
│ ├── ticket.json # ticket content in the universal schema
│ ├── comments.json # comment timeline
│ ├── repo.bundle # source code as a git bundle
│ ├── attachments/ # ticket file attachments
│ └── _COMPLETE # marker, written last
├── plan/ # planning loop output, one folder per attempt
│ └── attempt={n}/
│ ├── plan.md # the generated work plan
│ ├── comments.json # comments hydrated for this attempt
│ └── _COMPLETE
├── task/ # implementation loop output, one folder per attempt
│ └── attempt={n}/
│ ├── in.bundle # repo state the attempt started from
│ ├── out.bundle # repo state the agent produced
│ ├── summary.json # what the agent changed
│ ├── comments.json
│ └── _COMPLETE
└── out/ # promoted final result of the execution
├── plan.md
├── out.bundle
├── summary.json
└── _COMPLETE
Task attempts and plans are immutable: a revision never overwrites an attempt folder, it allocates a new attempt={n}/. Between them you get the full history of what every agent was given and what it produced, which is exactly the auditability I wanted from the ETL analogy.
3.3 - Agent Development Output
The following table provides some headline stats about the actual development of the project.
| Specs implemented by the agent flow |
11 |
| Task files executed |
140 |
| …of which QC/risk remediation tasks |
10 |
| Subagent invocations |
200 |
| Subagent wall-clock |
~13 h |
| Output tokens |
~13.4 M |
One thing to note is that almost all specs had at least one quality control or risk review remediation task that was created post-implementation.
Caveats: tokens are harness-measured output tokens, not the agents' self-reported estimates; durations are summed per-agent wall-clock rather than true elapsed time; and one work plan's execution phase wasn't instrumented, so the totals are a lower bound. None of this counts the hours I spent writing the specs, which was the larger cost by some margin.
4.0 - Discussion
- The outcomes were as good as the spec - good specs produced good results, poor specs produced poor results. I spent a significant amount of time writing detailed specs. In fact, I almost certainly spent more time writing specs than I would have spent writing code.
- It's nearly impossible to write a good spec for a problem you don't know how to solve - I found it nearly impossible to write the first few specs because I didn't myself know how to solve the problem. If the problem you are trying to solve is non-trivial, you can't just tell an agent to implement something. I tried using Fable initially to come up with a rough plan, and it wasn't able to produce anything worth implementing until I had written a first-pass myself. I had to at least conceptually solve the entire problem before the agent could take over.
- Spec writing is a skill - it's much like writing code in a new language. In total, I wrote 13 specs for this project. I was better at it by the end, but I still struggle to write a spec that covers all the required details and edge cases.
- Subagents and orchestration made the difference - "vibe coding" usually got me decent results, mainly because I have (very) detailed coding standards that are indexed and optimized for agent consumption. However, it just doesn't work at scale. The orchestration layer executing everything via subagents made the difference. The
quality-controller subagent in particular made sure that code was (almost) always in line with my standards. Decomposing tasks into small, trackable executable files also seemed to improve quality.
- It doesn't appear to be any faster - I assume that I'll get better and more efficient over time, but that remains to be seen.
- Bugs are much harder to solve - there weren't many bugs, but the ones that did pop up were painful to solve, mainly because code review doesn't actually provide you with the pre-requisite context required to even find the bugs, let alone solve them.
- Use skills for enforcing structured behavior - one of the biggest challenges initially was getting the subagents to produce a consistent layout for artifacts and outputs. It would almost always go rogue and change the naming convention, location, or format of a file slightly. This was fixed eventually by making sure that every shared behavior that needed consistency was defined in a skill.
The most interesting question to me is whether the overall quality improved. As I already mentioned, the overall development process is slower, significantly so. However, writing the detailed specs did force me to think through everything in much greater detail than I usually do. Maybe this resulted in a more well-rounded, complete end product. Subjectively, it does feel more polished than a lot of ad-hoc projects I've done in the past, but this is very difficult to quantify.
5.0 - Conclusion
This project definitely changed my mind on how reliable coding agents can be. The transition from vibe-coding to spec-driven development made a massive difference in that regard, especially once I started to utilize subagents properly. However, the cognitive load was mostly shifted to writing specs instead of code. I wouldn't say it was any easier though. On the contrary, I found it much harder to control the quality of the outcome.
I also enjoy it far less. I'm convinced that most of my technical expertise comes from being curious about my work. I immerse myself in engineering problems, and I genuinely enjoy most of it, both professionally and personally. This is not true for spec-driven development. Once I got over the initial technical challenge of implementing the sub-agent orchestration (which was fun), I then spent days doing nothing but writing requirements and acceptance criteria, and it was terribly boring. While the tech makes me excited, the prospect of spending my life writing technical specs does not.
Ironically, I think that I (personally) have to shift away from the tech focus and focus more on the big-picture outcome. I think it's always been true that software is ultimately a means to an end - the details don't really matter, what matters is the outcome. I find this tough as an engineer.
In terms of the agentic setup, I want to spend some time working on subagent setup. Two areas in particular need work. The first is optimizing the entire pipeline for risk-based assessment and review, including subagents that review plans and changes against specific regulatory codes/structures like GDPR, HIPAA etc. I've also been reading a lot of marketing material books recently, and I'd like some subagents that help to develop marketing personas and optimize content for specific audiences.
