Who it's for
Engineering leaders about to let AI agents work on a real codebase.
You'll be able to
Pick a safe first step, and know what to measure before you give agents more room.
Start small, with one gate. Give agents more freedom only when your own numbers show it is safe.
Start with one gate
Your team probably has many checks already: written specs, tests first, AI review, a person who reads the code. The trouble is that they depend on people and agents following the steps. If a step is skipped, a failing change can still get in.
So the first step is not a new tool. It is to make the checks you trust run by themselves.
- Put tests, lint, type check, a secret scan and a package audit behind one command.
- Run that command in CI on every pull request.
- Protect the main branch and make the check required, for administrators too.
Git hooks give fast feedback, but a person or an agent can skip them. The required check in CI is the one that can't be skipped.
Fixed checks decide pass or fail. AI reviewers advise the person.
An AI checker's verdict should never block a merge on its own. Use it to point a person at what to read.
Keep a person on every merge
At the start, a person approves every change. That is a choice, not a gap. People decide what to build, approve the spec and the plan, and review each change. Agents write the code and the tests. The checks run on every change.
| Step | The person | The agent | The checks |
|---|---|---|---|
| Describe and plan | Approves | Asks questions, writes the spec and plan | Nothing |
| Build | Takes over when the agent keeps failing | Writes code and tests | Run on every change |
| Review | Approves or rejects, and is responsible | Gives advice | Must pass first |
Set a retry limit: after a few failed fix rounds, the agent stops and hands the task to a person. Treat the number as a starting policy and measure it.
Write the data rules before the data
Before any agent writes a log, decide three things. Which data is public, internal or secret. What may be logged, and what must be removed first. What may be sent to a model provider. Set these before the first log is written, not after.
Measure before you widen
Decide what you will watch: how fast you ship, how often a change breaks, and what each piece of work costs. Write unknown as unknown, never as zero. Cost figures reported by AI tools are estimates; label them so.
Choose tools the same way. State your limits first: how much paid model use you have, and whether code may leave your machines. Then run the same test on each setup you are considering, and decide from your own results.
A fast lane for low-risk changes, or longer runs with nobody watching, can come later. Open them only by a recorded decision, when the numbers support it.
- Off
A person approves every change. The checks run on every change.
A person starts the trial. - Trial
Each low-risk change is scored and recorded: would the lane let it through? A person still approves every change. You watch the numbers.
A person opens it by a recorded decision, when the numbers support it. - Open
A fast lane: low-risk changes that pass every rule may merge without a person. The checks still run on every change.
StopAnyone on the team can stop it at any time. It goes back to the trial, or off.
Freedom for agents is earned in three stages, one lane of low-risk work at a time.
- Off: A person approves every change. The checks run on every change. To move on: A person starts the trial.
- Trial: Each low-risk change is scored and recorded: would the lane let it through? A person still approves every change. You watch the numbers. To move on: A person opens it by a recorded decision, when the numbers support it.
- Open: A fast lane: low-risk changes that pass every rule may merge without a person. The checks still run on every change.
Anyone on the team can stop it at any time. It goes back to the trial, or off.
Try it
Pick one repository and one month.
- List every check your team runs before a merge. Put the ones that give a clear pass or fail behind one command, required in CI.
- Write down who approves specs, plans and merges.
- Choose three numbers to track. At the end of the month, look at which ones you could actually measure.