Lesson 1 of 5

Testing: Unit, Feature and HTTP

"Testing shows the presence, not the absence of bugs." — Edsger W. Dijkstra, NATO Software Engineering Conference (1969)

A test suite is a program that runs your program and complains. There is no other magic: an assertion is an if and a counter, a test double is a class implementing an interface, and a database test is a transaction that gets rolled back. Everything a framework adds is ergonomics on those three ideas.

What is worth measuring is not how to write a test — that is a syntax you learn in an afternoon — but what each kind costs, what it covers, and the ways a passing suite lies to you.

A runner is assertions and a counter

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

Fewer than forty lines and you have a usable test runner. That is what PHPUnit is at its core, with several hundred assertions, discovery and reporting built around it.

The assertion table is the practical part. assertSame is strict and assertEquals is loose, and given how much of this volume has been about '200' not being 200, the loose one hides exactly the bug class you most want caught. Reach for the most specific assertion available: every assertion's real product is its failure message.

An analogy

Think about how a new bridge is checked before it opens. Somebody tests a single weld in a laboratory: a small specimen, a precise load, a fast and exactly repeatable result that tells you about that weld and nothing else. Somebody else tests an assembled section on site, which takes far longer, needs the actual steel and the actual bolts, and tells you whether the parts fit together. And finally lorries are driven across the finished bridge — slow, expensive, impossible to repeat very often, and the only test that answers the question anybody actually cares about. Notice that none of these replaces the others and the trade between them is always the same: the more of the real system a test includes, the more it tells you and the less often you can afford to run it. Notice also what the lorry test cannot do — when it fails, it does not tell you which weld. That is the whole structure of a test suite, and the reason arguments about "unit versus integration" are usually arguments about where somebody wants to sit on that curve.

What each level costs, and covers

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

A unit test costs about 0.001ms here, integration around 0.02ms and HTTP a little more — more than an order of magnitude apart, though the exact multiple swings between runs because the unit test sits near this runtime's clock floor. And it understates a real suite, which adds a socket, framework boot and factory hydration.

The suite-shape block is the consequence: costs multiply by test count, so the distribution decides whether the suite runs in seconds or minutes, and a suite taking minutes stops being run before every commit. That is how a test suite fails — not by being wrong, but by becoming something people skip.

What the table does not say is that unit tests are better. They cost less and cover less; only the HTTP test catches a broken route, a middleware ordering bug or a serialisation mistake. The cost curve tells you how many of each you can afford, not which are worth having.

A little history

Modern unit testing starts with Kent Beck's SUnit, written for Smalltalk in 1994, whose central insight was social rather than technical: tests are worth writing only if running them is cheap enough that you will do it constantly. The 1997 Java port as JUnit, with Erich Gamma, spread the idea, and every xUnit since — including Sebastian Bergmann's PHPUnit, released 2001 — is the same design: test classes, setup and teardown, assertions, a runner. What is worth knowing is that the "test pyramid", many unit tests at the base and few end-to-end at the top, is newer and shakier than its ubiquity suggests. Mike Cohn described it in Succeeding with Agile in 2009 as a rough heuristic, and it has been argued about since — the "testing trophy" reweights towards integration tests on the grounds that they catch more real defects per test, and plenty of teams find a suite of heavily-mocked unit tests reaches high coverage while verifying almost nothing about whether the system works. The durable part is the cost curve rather than the shape: tests including more of the system tell you more and cost more, so you buy fewer. Which ratio is right is an empirical question about where your bugs come from, not a rule.

The transaction is what makes database tests repeatable

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

Two tests, the same starting count, because each ran inside a transaction rolled back in a finally. That is the whole of RefreshDatabase — the transaction from Part VI ch1, applied per test.

The limitation matters more than the technique. Uncommitted data does not exist to any other connection, so the moment a test involves a real browser or a queue worker, the trick stops working and you need a strategy that commits. That is the classic "my end-to-end test says the record is missing", and it is not a bug in anything.

Test doubles, and the one distinction that matters

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

Five kinds, and the meaningful distinction is the last. A spy records and you assert afterwards, so a failure points at your assertion. A mock carries the expectation into the run, so a failure points at the call that broke it — a better stack trace, at the cost of coupling the test to how the code works.

So heavily-mocked tests break on refactors that change nothing observable, because they assert on interactions. Spies and fakes assert on outcomes and survive. Laravel's Mail::fake() and Queue::fake() are fakes with assertion helpers, which is why they feel so much less brittle.

From my notebook

While probing flakiness I wrote what I intended as an illustration of a time-dependent test — comparing the year of a fixed timestamp against date('Y') — and it did not illustrate the problem, it had the problem: the assertion failed, on this run, because the fixture's year is 2023 and the current year is not. I had written a deliberately brittle test to demonstrate brittleness and it broke immediately, which is a better demonstration than the one I planned and also slightly humbling. The genuinely useful part is what it says about when this class of bug surfaces. A test comparing against "now" passes every single time it is run in the window its author was thinking about, and starts failing on a boundary they never considered — the year rolls over, the clock hits a month end, a date-of-birth fixture quietly ages past eighteen. So the failure arrives detached from any change: nobody touched the code, CI went red overnight, and the first hypothesis is always infrastructure. I have started treating any use of now(), time() or date() inside a test as a defect regardless of whether it is currently passing, on the grounds that "currently passing" is not a property of the test but of the calendar.

How a passing suite lies

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

Four ways a green suite tells you nothing, three of them about determinism.

The time case is live in that output: a fixture from 2023 compared against the current year, failing today, with nobody having touched any code. Tests that consult the real clock pass while the author's assumptions hold and then break on a boundary detached from any commit — which is why the first hypothesis is always infrastructure.

The order case is shared state. Statics, container singletons, files and auto-increment ids all survive between tests, so a suite passes in written order and breaks when someone inserts a test above — or when the runner parallelises.

The fourth catches careful people, and it is the free-pass problem from this volume's own hands-on discipline: a loop asserting over an empty collection makes zero assertions and reports success. Assert the count, then the contents.

The big question

What is a test suite actually for, given that it demonstrably cannot prove code correct? Three answers get offered and they are not equally good. The first is "catching bugs", which is weakest: a test catches what its author thought to check, so by construction it finds the bugs you already imagined. The second is "enabling refactoring", much stronger — a suite's real product is confidence to change code you no longer remember, which is why tests asserting on behaviour beat tests asserting on interactions: the first survive a rewrite of the internals and the second break on every one. The third and most underrated is "specifying intent": a test is the only place expected behaviour is written down in a form that cannot go stale, because a stale comment is ignored and a stale test is red. That framing changes what you write. If tests catch bugs, more coverage is better and mocking everything to reach it is rational. If tests enable refactoring and specify intent, a hundred tests asserting on mock interactions are a liability — they break when you change the implementation while telling a future reader nothing about what the system promises. The suites I have found most valuable were not the highest-coverage ones. They were the ones where reading the test names told you what the application did.

In the real world

The most damaging thing a test suite does is not fail — it is become slow enough to skip. The progression is predictable and nobody decides on it. A suite that runs in twenty seconds gets run before every commit. At two minutes it gets run before pushing. At eight minutes it gets run in CI only, which means the feedback loop moves from seconds to the length of a pipeline, and developers start pushing speculative fixes to see what CI says. At twenty minutes people begin merging on a green-ish build and "we'll fix the flaky ones later", and at that point the suite has stopped being a safety net and become a tax. What makes it worth naming is that no individual decision caused it: every test added was reasonable, and the one that tipped it over was no worse than the rest. The defences are cheap early and expensive later — keep fast and slow suites separately runnable, treat suite duration as a number somebody watches, and treat a flaky test as broken and fix or delete it within days. A red build that is sometimes wrong trains everybody to ignore red builds.

Common Mistakes

Using assertEquals where assertSame is meant. The loose comparison hides exactly the string-versus-int bugs this volume keeps finding.
Consulting the real clock. A test comparing against now() passes until a boundary crosses, then fails with no commit to blame. Inject a clock.
Sharing state between tests. Statics, singletons, files and auto-increment ids survive, so the suite passes in written order and breaks on insertion or parallelisation.
Using unseeded random data. A factory will eventually produce the value that breaks an assumption, once in a few hundred runs. Seed it and print the seed.
Asserting inside a loop without asserting the count. An empty collection makes zero assertions and reports success — the free-pass problem again.
Mocking everything. Interaction assertions break on every refactor even when behaviour is unchanged, which inverts what the suite is for.
Expecting transactional tests to work with a second process. Uncommitted rows do not exist to a browser or a queue worker; those tests must actually commit.
Nesting transactions accidentally. Code under test that opens its own transaction inside the test's transaction needs savepoints — PDO has only one.
Letting the suite get slow. Past a few minutes it stops being run before commits, and a suite people skip has stopped working regardless of what it asserts.
Tolerating a flaky test. It trains the team to ignore red builds, which costs more than the test was ever worth. Fix it or delete it.

Interview Corner

Interview Corner — the same idea at three levels. Answer yours; recognise the one above it.

Warm-up

What is the difference between a unit test and a feature test?

Model answer — How much of the system is included. A unit test exercises one piece in isolation — a calculation, a class with its dependencies replaced — so it is fast and tells you precisely where a failure is. A feature or HTTP test sends a request through routing, middleware, the controller, the database and serialisation, so it is slower and tells you whether the parts actually work together, but a failure does not say which part. I measured more than an order of magnitude per test between the two here, which is why suites usually have many of the first and fewer of the second — not because unit tests are better, but because the cost decides how many you can afford.

Intermediate

A test passes alone and fails in the suite. What causes that?

Model answer — Shared state surviving between tests. The usual sources are static properties, container singletons that persist across tests in the same process, files written by one test and read by another, and assertions on auto-increment ids that only hold if that test ran first. The diagnostic is to run the suite in a random order — most runners support it — because that turns an intermittent failure into a reproducible one. The related case is the reverse, a test that passes in the suite and fails alone, which usually means it depends on data another test created. Both come from the same root, which is that the tests are not independent, and the fix is to make each test establish everything it needs.

Advanced

When would you use a mock rather than a fake, and what does it cost?

Model answer — A mock carries expectations into the run and fails at the offending call, so the stack trace points at the line that violated the contract rather than at your assertion — that diagnostic precision is the real reason to use one, and it matters most when the interaction itself is the behaviour you care about, like "we must not charge the card twice". What it costs is coupling to implementation: mocks assert on how the code works, so they break during refactors that change nothing observable, and a suite of heavily-mocked tests becomes a tax on exactly the changes the suite was meant to enable. Fakes and spies assert on outcomes instead and survive refactoring, which is why Laravel's Mail::fake() and Queue::fake() — which are fakes with assertion helpers — feel so much less brittle. My default is a fake, with mocks reserved for the cases where the call itself is the requirement.

micro1Practical: the levels and what each covers.GoogleDeepest — determinism and what a green suite does not prove.MetaTest doubles and coupling to implementation.AmazonSuite health as an operational concern.

Hands-on

Model the testing rules this chapter measured. Implement levelOf, classifying a test by what it touches; suiteMs, the cost arithmetic; doubleFor, choosing a test double; isDeterministic, the flakiness sources; and assertionsMade, the empty-loop trap. A stub scores zero, so a passing run means the model is real.

PHP — Try it Yourself

Loading editor…

Output

Click Run to execute the code.

Summary

A test runner is assertions and a counter. Everything a framework adds is discovery, reporting and ergonomics on top of that.
Prefer the most specific assertion, and assertSame over assertEquals — the loose one hides the type bugs you most want caught.
Integration and HTTP cost more than an order of magnitude above a unit test here, and that understates a real suite, which adds a socket, framework boot and factories.
The cost curve decides how many of each you can afford, not which are worth having — only the HTTP test catches a broken route or middleware order.
RefreshDatabase is a transaction per test, rolled back — the same mechanism from Part VI, applied per test.
Transactional tests break with a second process, because uncommitted rows do not exist to a browser or a queue worker.
A spy fails at your assertion; a mock fails at the call. The mock gives a better stack trace and couples the test to implementation.
Reading the real clock makes a test fail on a calendar boundary, detached from any commit — demonstrated live in this chapter's own block.
Shared state makes the suite order-dependent; running in random order converts an intermittent failure into a reproducible one.
A loop asserting over an empty collection asserts nothing and passes. Assert the count first.

Next: queues and background work — moving what must not happen during a request out of it, and what changes once the work runs somewhere else.

Editorial reflection

The thing I find most interesting about testing is that its hardest problems are not technical. Writing an assertion is trivial; deciding what deserves one is a judgement, and the judgement depends on a question most teams never answer explicitly — what is this suite for? A suite written to catch bugs looks completely different from one written to enable refactoring, and a codebase containing both kinds tends to have been written by people who disagreed without noticing. That unstated disagreement is, I suspect, behind most arguments about coverage percentages and about mocking, which are proxy fights for a question nobody put on the table. The other observation worth recording is that this chapter's failure modes are almost all the same shape as the rest of the volume's. A test that asserts nothing passes; a test that reads the clock fails without a cause; a suite that gets slow stops being run. None of these announce themselves, all of them look like success, and every one is caught by making a number visible rather than by trying harder — the assertion count, the suite duration, the seed. I did not plan for this Part to rhyme with the earlier ones quite so exactly, and I have stopped being surprised when it does.

Enrol to track progress

Q&A · 0

Enrol to ask questions and join the discussion.

No questions yet — be the first to ask.