Testing the Mock: What Interaction Assertions Prove About the Stand-In

On 14 March 2024, a release candidate for a billing service went green. The suite included a test that did roughly this: construct a payment processor with a mocked gateway, call makePayment, then assert verify(gateway).charge(card, amount). The mock returned true. The assertion passed. The ticket said the charge path was covered. The release went out.

What users got was a charge in the wrong unit. The processor passed dollars; the real gateway expected cents. Every interaction assertion in the suite passed because the mock never validated units — it only recorded that charge had been called with whatever arguments the system under test supplied. The instrument reported a verified interaction. The ticket reported coverage. The users got a hundredfold overcharge.

This is not a story about a bad mock. It is a story about what an interaction assertion can and cannot certify, and why release owners keep mistaking one for the other.

What the green actually certifies

Martin Fowler’s distinction is the place to start. In state verification, a test determines correctness by examining the state of the system under test and its collaborators after the method is exercised. In behavior verification, the test checks whether the system under test made the correct calls on the collaborator. Mocks are the double that insists on the second style: they are, in Fowler’s phrasing, “objects pre-programmed with expectations which form a specification of the calls they are expected to receive” (Mocks Aren’t Stubs).

Read that sentence again with a release decision in mind. The expectations are pre-programmed by the test author. The mock does not know the real gateway’s contract. It knows the script the test writer wrote. When the assertion passes, what has been proven is that the system under test conformed to the test author’s model of the collaborator — not to the collaborator itself.

This is the circularity at the center of interaction testing. The test author writes down what calls should happen. The mock enforces that script. The assertion confirms the script was followed. The real dependency is never consulted, so its actual behavior — units, ordering, error semantics, idempotency — is outside the scope of what the assertion can prove.

The double’s role determines what the assertion means

The vocabulary matters here, not as pedantry but because the words describe different evidence. Fowler’s taxonomy separates dummies, fakes, stubs, spies, and mocks, and notes that “of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification.” A stub returns canned answers. A fake has a working but simplified implementation. A spy records calls. Only a mock is built around expectations that decide pass or fail.

Microsoft’s .NET testing guidance makes the same point from the other direction: “The main thing to remember about mocks versus stubs is that mocks are just like stubs, except for the Assert process. You run Assert operations against a mock object, but not against a stub” (Unit testing best practices for .NET). The same page warns that the literature uses these terms inconsistently, which is why a test named mockOrder may in fact be a stub that never participates in an assertion.

For a release owner, the practical consequence is this: when you see an interaction assertion, ask what kind of double it is running against. If the double is only returning canned values and the assertion checks that it was called, you are asserting on a script you wrote. If the double is a fake with a working implementation, the assertion may be checking behavior the fake actually models — but only to the fidelity of the fake.

The fidelity gap the assertion cannot see

Google’s Software Engineering at Google chapter on test doubles defines fidelity as “how closely the behavior of a test double resembles the behavior of the real implementation that it’s replacing,” and states plainly that if the behavior of a double significantly differs from the real implementation, tests using it “likely wouldn’t provide much value” (Test Doubles). The chapter’s example is a database double that ignores writes and always returns empty results — every test passes, and the system is broken.

Interaction assertions are especially exposed to this gap because they never exercise the real contract. A state-based test against a fake at least observes what the fake did with the data. An interaction assertion observes only that a call was made. If the real dependency would have rejected the arguments, transformed them, or interpreted them in different units, the interaction assertion cannot know. The double was never asked.

The billing example is the canonical shape: the mock returns success, the assertion confirms the call, and the unit mismatch lives entirely in the space between the stand-in and the real gateway. No amount of interaction assertion density closes that space. Only a test that exercises the real contract — a contract test, an integration test against a real or high-fidelity fake — can.

The coupling cost release owners feel as red builds

There is a second cost, and it shows up as suite noise rather than production incidents. Interaction tests are coupled to the call sequence. When the system under test is refactored to call its dependencies differently — even if observable behavior is unchanged — the assertions fail.

One practitioner account describes exactly this: after changing the internals of a message-sending class, “as soon as I changed the internal implementation, my test code broke. More specifically, the calls to mocks. My tests were quite coupled to the internals of the MessageSender. The tests knew exactly which methods had to be called in its dependencies” (Tests breaking because of mocks, what to do?). The author’s resolution was to treat the test as an interaction test and accept the breakage, or to restructure the code so new behavior could be added without rewriting existing tests.

Google’s experience is described in similar terms. The Software Engineering at Google chapter records that mocking-framework tests were easy to write but “required constant effort to maintain while rarely finding bugs,” and that “the pendulum at Google has now begun swinging in the other direction, with many engineers avoiding mocking frameworks in favor of writing more realistic tests.” That is a qualitative account of organizational experience, not a measured rate, and it should be read as such. But it matches what release owners see: a suite that goes red for refactors that changed nothing users can observe.

Steelmanning the interaction assertion

The counterargument deserves a fair hearing, because interaction assertions are not a mistake. They are the right instrument for a specific class of claim.

Consider a side effect the system under test is responsible for: sending a notification, initiating a charge, publishing an event. The real dependency is slow, non-deterministic, or costly to run — a payment gateway that charges real cards, a chat service that pages real people. In that situation, the only way to know the system attempted the side effect is to observe the call. A state-based test against a fake could work, but writing a high-fidelity fake is itself expensive, and the fake can drift from the real service. The interaction assertion is cheap, fast, and directly checks the thing the system is responsible for: that it tried.

Google’s own guidance acknowledges this. The Software Engineering at Google chapter describes interaction testing as “a way to validate how a function is called without actually calling the implementation of the function,” and notes that a test should fail if the function isn’t called the correct way — not at all, too many times, or with the wrong arguments. The technique has a legitimate scope. The problem is not the technique; it is the claim attached to it.

The steelman, then, is this: an interaction assertion is a narrow, named claim — “the system under test attempted this side effect with these arguments” — and within that scope it is sound evidence. What it cannot do is stand in for a contract test of the real dependency. When a release decision treats the interaction assertion as proof that the integration works, the claim has been stretched past what the instrument can support.

A triage rule for release-blocking suites

The recommendation is not to delete interaction assertions. It is to make the claim explicit and check whether anything else covers the assumption.

For each interaction assertion in a release-blocking suite, write down the real-dependency behavior it assumes. In the billing example, the assumption is: the real gateway accepts the amount in the unit the system passes, and returns success for valid charges. Then ask: which test exercises that assumption against the real dependency or a high-fidelity fake? If the answer is none, the assertion is a script check, not a contract check. It belongs in the suite, but it does not belong in the release argument as evidence that the integration works.

This is a small procedural change with a large effect on release conversations. It converts a green checkmark into a named claim with a named gap. The gap may be acceptable — the contract may be covered by a separate integration suite, or the real dependency may be stable enough that the risk is low. But the gap is now visible, and the release owner can decide with it in view rather than discovering it in production.

The artifact to cross-examine

The mock is an artifact like any other in the quality record: a test suite, a CI config, a dashboard, a status page. It carries a claim. The claim is written by the test author, enforced by the framework, and reported as green. The release owner’s job is to read the claim, not the color.

When the next release candidate goes green on an interaction assertion, the question is not “did the mock get called?” It is “who verified the mock’s model of the collaborator?” If the answer is a contract test, the assertion is part of a chain of evidence. If the answer is nobody, the assertion is a check on internal wiring wearing the costume of a contract check — and the costume is convincing enough that it will keep passing right up until users find the gap.

FAQ

Does a passing interaction assertion prove the integration works?

No. It proves the system under test made the calls the test author scripted on the double. Whether the real dependency would accept those calls — same units, same argument shapes, same error semantics — is outside what the assertion exercises. That requires a test against the real dependency or a high-fidelity fake.

Is interaction testing always the wrong choice?

No. It is the right instrument when the system under test is responsible for a side effect that is costly or non-deterministic to run for real, and when the claim is kept narrow: “the system attempted this call with these arguments.” The problem is stretching that claim into a general proof of correctness.

How do I tell a mock from a stub in an existing suite?

Look at whether the double participates in an assertion. Microsoft’s guidance puts it directly: you run Assert operations against a mock, but not against a stub. If the double only returns canned values and nothing asserts on its calls, it is a stub regardless of its variable name. If the test verifies calls on it, it is functioning as a mock.

What about spies and fakes — do the same limits apply?

The fidelity limit applies to any double. A spy records calls but does not decide pass or fail on its own; a fake has a working implementation whose fidelity to the real dependency is the question. Interaction assertions on a spy or a fake still prove only what the double recorded, and the double’s model of the real dependency still needs independent verification.

Should I remove interaction assertions from my suite?

Not by default. The recommendation is to make the claim explicit: for each assertion, name the real-dependency behavior it assumes and identify the test that exercises that assumption. If no such test exists, the assertion is a script check — keep it if it is useful, but do not count it as contract evidence in the release decision.