Testing in the AI EraTesting in the AI Era

I love testing. The whole idea of having code that runs other pieces of code to verify that what it does is correct ticks all the boxes for me. And quite literally, since you should see a lot of ✅ (hopefully!) when running the tests. However, testing in the correct way is quite complex and in this post I'll show you how to test effectively from more than a decade of experience.

expect(1 + 1).toBe(2);

#Deciding how to test

There's a great variety of types of tests:

• Unit: Tests one piece of logic in isolation, fast and focused.
• Integration: Tests that several pieces work together correctly, like your code against a real database.
• E2E: Tests the whole system from the outside, like a user would.
• Characterization: Captures what legacy code currently does, so you can refactor without changing behavior.
• Performance: Measures speed and load against a defined threshold.
• Property-based: Declares a rule that must always hold and generates hundreds of inputs to try to break it.
• Mutation: Introduces small bugs into your code to check if your tests catch them.
• Regression: Locks in a fixed bug so it can never come back.

In this post we'll focus on the principles that actually make testing robust across all types of tests. However, before writing any line of code (or telling your AI agent to), we need to first think what would be most beneficial to us at this point in time.

For example, E2E testing is great to begin with if you have no prior tests, that way you can verify that core functionality works as it should (that is also commonly referred as happy paths!). The challenge in projects that have no prior tests is that although testing on its own is quite simple, the challenging part is having code that is actually easy to test.

“

Testing is easy, the challenging part is having testable code

”

Starting with E2E tests circumvents that challenge while actually providing value to the project by testing the resulting artifact rather than the internals.

Then, a natural step is to move towards unit testing and then integration tests. Sometimes in that process we can add characterization tests to validate the current behavior of something.

For projects that have no testing, this is the order I follow:

1. E2E
2. Characterization
3. Unit
4. Integration
5. Mutation
6. Property-based
7. Regression
8. Performance

If the project has testing, then we need to think about where we currently are and what is valuable for the project, and I'm afraid only you can answer that question.

#Deciding what to test

Once we've figured out how to currently test our application, we need to figure out what to test. Should we aim for 100% coverage? Only test the most essential functionality? Only test files with an odd number of letters?

// Production code
function applyDiscount(price: number, percent: number): number {
  return price - (price * percent) / 100
}

// Test with 100% coverage... that verifies nothing
test('applies discount', () => {
  applyDiscount(100, 20) // executed, never asserted
})

// Test with the same coverage, but real verification
test('applies discount', () => {
  expect(applyDiscount(100, 20)).toBe(80)
})

To me, I lean towards testing what makes sense. Disappointing answer, right? However, I think it is the right answer. I've been in projects close to 100% test coverage, and still I didn't trust the code. The tests would be testing internal code rather than the functionality, or tests would be duplicated.

Acquiring taste of what to test takes time.

Here is an example of a great test and a poor test:

// ✅ A great test: verifies behavior through the public API
describe('Cart', () => {
  it('applies a 10% discount to the total when a valid coupon is used', () => {
    const cart = new Cart();
    cart.add({ name: 'Keyboard', price: 100 });
    cart.add({ name: 'Mouse', price: 50 });

    cart.applyCoupon('SAVE10');

    expect(cart.total()).toBe(135);
  });
});

// ❌ A poor test: coupled to internals, verifies nothing the user cares about
describe('Cart', () => {
  it('works', () => {
    const cart = new Cart();
    const spy = vi.spyOn(cart as any, 'recalculate');

    cart.add({ name: 'Keyboard', price: 100 });

    expect(spy).toHaveBeenCalledTimes(1);
    expect((cart as any).items.length).toBe(1);
    expect((cart as any).discount).toBe(0);
  });
});

The difference is quite big as the project grows.

So, always focus on testing the public API and how a user would actually use the code.

#AAA

In the previous tests you might have seen a newline in what seems arbitrary places.

describe('Cart', () => {
  it('applies a 10% discount to the total when a valid coupon is used', () => {
    const cart = new Cart();
    cart.add({ name: 'Keyboard', price: 100 });
    cart.add({ name: 'Mouse', price: 50 });
    // Here
    cart.applyCoupon('SAVE10');
    // Here
    expect(cart.total()).toBe(135);
  });
});

Well, it's not arbitrary.

I do this to group under the AAA pattern, which means the Arrange Act and Assert.

1. Arrange: This is where we prepare the mocks, create the instances, and all necessary data for the tests
2. Act: Run the code that actually needs to be run for the test to test what's required (also known as Subject Under Test or SUT)
3. Assert: Make the necessary verifications. I tend to have one verification per test

If the arrange group is duplicated across tests I create a setup function and move it to the bottom of the file. I use an object to configure the setup if necessary. I prefer this approach rather than the beforeEach because it results in less code:

function setup() {
  const cart = new Cart();
  cart.add({ name: 'Keyboard', price: 100 });
  cart.add({ name: 'Mouse', price: 50 });
  return cart;
}

Vs

describe('Cart', () => {
  let cart: Cart;

  beforeEach(() => {
    cart = new Cart();
    cart.add({ name: 'Keyboard', price: 100 });
    cart.add({ name: 'Mouse', price: 50 });
  });

  it('applies a 10% discount to the total when a valid coupon is used', () => {
    cart.applyCoupon('SAVE10');

    expect(cart.total()).toBe(135);
  });
});

The first option is more versatile and configurable, since we can just pass through parameters new options. We also favor immutable code:

function setup({ items = [{ name: 'Keyboard', price: 100 }, { name: 'Mouse', price: 50 }] } = {}) {
  const cart = new Cart();
  items.forEach(item => cart.add(item));
  return cart;
}

#TDD

If you pair all of the above with Test Driven Development you get, in my opinion, the most reliable way of building software. TDD is deceptively simple. You repeat three steps, in this exact order:

Let's see all of this in action with the classic FizzBuzz kata.

The problem: write a function that receives a number and returns:

- "fizz" if the number is divisible by 3
- "buzz" if the number is divisible by 5
- "fizzbuzz" if it's divisible by both
- The number itself, as a string, otherwise

Let's start by creating the first test:

it('returns fizz when divisible by 3', () => {
  expect(fizzbuzz(3)).toBe('fizz');
});

function fizzbuzz(number: number): string {
  return 'fizz';
}

Yes, really. The test passes, and that's all that matters right now. It feels silly, but it forces the tests to drive the implementation, not our assumptions. Each new test provides new assumptions:

it('returns the number as a string when not divisible by 3 or 5', () => {
  expect(fizzbuzz(1)).toBe('1');
});

function fizzbuzz(number: number): string {
  if (number % 3 === 0) return 'fizz';
  return String(number);
}

The same for buzz, until the final test:

it('returns fizzbuzz when divisible by both 3 and 5', () => {
  expect(fizzbuzz(15)).toBe('fizzbuzz');
});

Red! fizzbuzz(15) returns 'fizz' because the first if wins. The tests just caught a real bug in our design:

function fizzbuzz(number: number): string {
  if (number % 15 === 0) return 'fizzbuzz';
  if (number % 3 === 0) return 'fizz';
  if (number % 5 === 0) return 'buzz';
  return String(number);
}

All green. Now, and only now, we refactor with total confidence, because any misstep turns a test red instantly.

Each implementation forced the next test to be more specific, and each test forced the code to be more general. That is TDD.

#Testing in the AI era

So, why the title of this post? Because everything above just became more important, not less.

Agents can generate more code in an afternoon than we used to write in a week. Code is now cheap. Trust is, however, the problem challenge. And a solid test suite is the most efficient way to get trust.

When an agent writes code, the tests become the contract: the agent runs them, reads the failures and iterates until everything is green. This is also why TDD and agents pair so well. A failing test is the best prompt you can give an agent: it's precise, it's a computational check, and it leaves no room for interpretation.

But (there's always a but) agents are also shockingly good at gaming tests: assertions that assert nothing, mocking the very thing under test, or deleting a failing test to "fix" the build. Every antipattern we've seen in this post, produced at machine speed.

Which means your role shifts: your testing taste is now your most valuable skill. Review the tests an agent writes with more care than the production code, because those tests are the specification the next agent will follow.

To make that taste transferable, I've distilled everything in this post into a Claude Code skill: create-test. My agents load it every time they write tests: public API, AAA, one assertion per test, setup functions. Copy it, adapt it, make it yours.

“

The agents write the tests. You provide the taste.

”

And you, how are you testing in the AI era? If you want more content like this, subscribe to the newsletter, and feel free to share your approach with me.