Tests first: prove it's broken, then fix it

Who it's for

Engineers who let agents write code and tests.

You'll be able to

Make an agent show a failing test before it writes the fix.

An agent can write a test that passes against almost any code. A test you have watched fail is one you can start to trust.

Why the order matters

A test written after the code tends to describe what the code does, bugs included. It passes the first time it runs, so you never learn whether it can fail at all. A test that never failed proves nothing.

So the factory turns the order around. The agent writes the test first, runs it, and sees it fail. Only then does it write the code that makes it pass.

First prove it's broken, then fix it.

What it looks like

Say the task is a total that adds tax. The test comes first:

def test_total_adds_tax():
    assert total(price=100, tax_rate=0.2) == 120

Run the tests. This one fails, because total does not exist yet. That failure is the proof that the test checks something. Now write total, run the tests again, and watch the same test pass.

01
Write the test
The agent writes the test first.
02
Run it
fails
It fails, for the right reason: total does not exist yet.
03
Write the fix
Only then does the agent write total.
04
Run it again
passes
The same test passes.
05
The gate runs everything
Every check, then the same review as any other change.

First prove it's broken, then fix it. A test that never failed proves nothing.

The same order holds for each new part: write the test, see it fail, then write the shape of the data, the logic and the route. For a new page, the test comes before the page too.

Bugs start as tests

When someone reports a bug, the first step is not the fix. It is a test that reproduces the report. It fails, as it should. Then the fix goes in, through the same gate and the same review as any other change.

What checks the order

Here is the honest part. The rule is written in the instruction file every agent reads: write the test before the code it covers. But the gate does not check the order. The gate checks that the changed lines are covered by tests. Coverage shows that a test ran a line. It does not show that the test ever failed.

One of the AI checkers looks at the order when it gets the list of commits. If the code came before its test, it warns. That is advice for the reviewer, not a block.

Try it

In your own repository, with any AI tool: