Why stop there? Why not delete your test methods too and just test in production?
Mocks are just test code, same as your test functions. And they’re necessary fur unit tests. If the thing you’re testing talks to another component without a mock, it’s now an integration test instead of a unit test.
Unit tests test the API surface of a component. They’re useful for ensuring a component adhere to its documented API contract. It’s also useful for testing how it reacts to edge cases. Integration tests are useful for ensuring components work together, and testing that data flows through the system properly. But integration tests make it harder to test various edge cases.
For example, if my component talks to yours and issues several requests to your component with callbacks, it may be that 99% of the time your component calls the callbacks in the same order. So I can’t really validate in an integration test that my component works properly if the callbacks are involved in a different order. By mocking your component in a unit test, I can test any order I want.
Unit tests also tend to be more deterministic. By mocking, I can eliminate sources of non-determinism from outside my component. As a trivial example, my component might need random numbers, so I might mock the random number generator to return a specific chosen sequence that I know will exercise different code paths.
Really the lesson that should be taught here instead of “don’t use mocks” is “unit tests aren’t sufficient, write integration tests too”.
> Really the lesson that should be taught here instead of “don’t use mocks” is “unit tests aren’t sufficient, write integration tests too”.
I completely agree with your comment.
I also think that people underestimate the kinds of scenarios unit tests can test; scenarios that are not possible or not practical to test by other means:
Unit tests are excellent for testing edge cases and scenarios that do not generally happen in normal production. It makes it easy to test things like 'how does this function behave if X returns some corrupted data?'. I feel like a lot of products don't test the failure cases at all, thinking 'that will never happen'. Mocking external services is a very straightforward way to test these kind of scenarios.
Just last week I was looking at a typical heisenbug that did not happen in 99%+ of production. The bug was masked by a combination of caching and the program flow. I was able to write a very short and specific unit test that was able to deterministically mimic the (bad) state the program was in and reproduce said bug.
There was however no way to properly test this using integration testing because it was relying on specific program state to manifest itself. Even if we had an integration test with the exact same data that caused the bug to appear, it would not be a deterministic test and future code changes might make it disappear even though the underlying bug is still there (or re-appeared).
> Why not delete your test methods too and just test in production?
This, but unironically.
Depending on your use case, any local tests, whether mocked or using a swarm of local containers attempting to represent production may be a far stretch from production reality. Put everything behind feature flags and test your contracts, then release to production regularly and test against live data and live services.
You don't test in production. By definition, prod is what you care about and don't want to break.
This is what preprod is for: an environment that stores, receives and processes the same data as prod. It replicates prod as closely as possible and errors or unexpected differences are investigated.
Basically, preprod and any not-prod environments cannot ultimately perfectly reproduce prod, and in a sufficiently complex system, the failure states cannot be predicted in a replicated environment. Instead, using strategies like rolling deployments, you can test on the actual environment you want to validate.
If set up properly, it doesn't really risk breaking prod.
Honestly I'd rather have all the units covered in isolation, and leave it up to my manual testing to catch any unexpected errors or bugs, then let our QA team do the same with their own Selenium based black-box testing.
If the contract between two units is flawed, you only have to worry about updating the behavior of one or both units. The unit tests make sure they work as designed.
It seems to abstract the problem for me to the level of only detecting missed integration issues. I don't need to worry about how those units do what they do, just that they do what they're designed, because the unit tests say so.
There's nothing I hate more than making a small change, and having to spend hours tracking down why 20+ tests that are written as integration tests rather than unit tests (with mocking).
I'd rather maintain the mocks when making changes, than have to spend all that time trying to figure out what complicated and huge, far reaching, thing is broken.
Mocks are just test code, same as your test functions. And they’re necessary fur unit tests. If the thing you’re testing talks to another component without a mock, it’s now an integration test instead of a unit test.
Unit tests test the API surface of a component. They’re useful for ensuring a component adhere to its documented API contract. It’s also useful for testing how it reacts to edge cases. Integration tests are useful for ensuring components work together, and testing that data flows through the system properly. But integration tests make it harder to test various edge cases.
For example, if my component talks to yours and issues several requests to your component with callbacks, it may be that 99% of the time your component calls the callbacks in the same order. So I can’t really validate in an integration test that my component works properly if the callbacks are involved in a different order. By mocking your component in a unit test, I can test any order I want.
Unit tests also tend to be more deterministic. By mocking, I can eliminate sources of non-determinism from outside my component. As a trivial example, my component might need random numbers, so I might mock the random number generator to return a specific chosen sequence that I know will exercise different code paths.
Really the lesson that should be taught here instead of “don’t use mocks” is “unit tests aren’t sufficient, write integration tests too”.