Stepping Down as Mockito Maintainer After Ten Years

Testing is hard. I’ve tried with AI today: No, it is still not capable of handling that kind of (straightforward) task (Using Claude).

You should be using it in rare cases when you want to verify very complex code that needs to be working with strict requirements (like calling order is specified, or some calls cannot be made during execution of the method).

Usually it is used for pointless unit tests of simple intermediary layers to test if call is delegated correctly to deeper layer. Those tests usually have negative value (they test very little but make any modification much harder).

This is kinda how we build software right? A little bit of "our logic" (calculation), represented as objects/actors/modules which "do things", but intermingled with million-LoC dependencies like databases and web servers.

After a while it gets frustrating setting up the robot hand and OCR equipment for each test case. Maybe it's just easier to test manually, or skip testing entirely.

At this point you can have an epiphany, and realise you only care about the numbers going in and out of the calculator, not the button pushes and pixels.

Mockito swoops in and prevents you from having that epiphany, by making it easier to keep doing things the stupid way.

Instead isolating the calculation from any IO, you can now write things like: when(finger.pushbutton(1)).then(calculator.setState(1)) when(calculator.setAnswer(3)).then(camera.setOcr(3))

(I've mostly worked in Java, but it seems like other languages typically don't let you intercept calls and responses this way)

Not mentioning of course needless copy-pasting dosens of members in the adapter. And it must be in prod code, not tests, even though it's documentation would say "Adapter for X, exists only for tests, to be able to mock X".

If tests (authored by someone else) break, I now have to figure out whether the breakage is due to the fact that not enough behavior was mocked or whether I have inadvertently broken something. Maybe it’s actually important that code avoid using “isEmpty”? Or do I just mock the isEmpty call and hope for the best? What if the existing mocked behavior for size() is non-trivial?

Typically you’re not dealing with something as obvious.

But when there are many tests where I instantiate a test fixture and return it from a mock when the method is called, I start to think that an in memory stub would have been less code duplication and boilerplate... When some code is refactored to use findByName instead of findById and a ton of tests fail because the mock knows too much implementation detail then I know it should have been an in memory stub implementation all along.

Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point.

We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds.

I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.

What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well.

Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously.

Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline?

Using mocks properly is a sign of a well-factored codebase.

More broadly, suppose foo() has an implementation that depends on Bar, but Bar is complicated to instantiate because it needs to know about 5 external services. Fortunately foo() only depends on a narrow sliver of Bar's functionality. Why not wrap Bar in a narrow interface—only the bits foo() depends on—and fake it?

I'm not a maximalist about test doubles. I prefer to factor out my I/O until it's high-level enough that it doesn't need unit tests. But that's not always an option, and I'd rather be flexible and use a test double than burden all my unit tests with the full weight of their production dependencies.

you could extend this to say 85% of the tome just write the code directly to prod and dont have any tests. if you broke something, an alarm will go off

To easily simulate failure cases, a range of possible inputs, bad data etc.

To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation.

Off the top of my head :-)

When used as a different way of writing an if else if can be a code smell. When used with an exhaustive list of an enum or sealed class prevents bugs in production.