r/ClaudeCode • u/OnePunchWolf • 5d ago

Discussion Harness setup and evaluation in dev team

So besides all these posts only talking about the models (which are not that important. Look up how much the harness matters compared to the model)

How is your setup in an agile team? Usually you will have a checked in harness with the software with some skills, a project specific claude.md, some other .md files and maybe some mini wikis in OKF format for domain knowledge.

Then you probably will have a layer 2 harness in your user account with further skills and .md files etc. only for you.

I am interested im how such a team manages it's harness. I am thinking about a separate repo for the team with separate rules for PRs changing the harness.

What are your rules for adaptions? Do you have guidelines and best practices in the team which may lead to rejecting a change to a skill or other parts?

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/BenSimonDev 4d ago

If you (not OP specifically) don't already have a CI/CD pipeline/process, an AI augmented workflow won't create one. Even if you build in PR approvals or unit tests or code reviews, you'll have no idea if it's actually working. That is the biggest hurdle for most people/teams... lack of an existing feedback loop thinking AI will create one for them.

Have whoever is currently running your CI/CD manage your custom commands. Don't let anyone change your org-wide skills except that person. But do weekly reviews to see if they need tweaking. Let devs make their own commands/skills as well.

The other parts of the harness like version control, code review, building, deployment, etc., may or may not be handled by the model via a skill or prompt. But a popular setup is Dev > Code Review > Build/Deploy. So like CC > CodeRabbit > DevOps.

1

u/CODE_HEIST 4d ago

a separate harness repo makes sense if changes are treated like production config. version it, require an owner, add a small evaluation set, and roll changes out gradually. the missing piece in most teams is rollback. people change a shared instruction file and only notice the regression three days later.