This is why I tend to lean toward the approach of only using mocks, stubs, etc. when absolutely needed. That tends to be at the places where the code is interacting with external components like databases or web services than with components that are within the current layer, such as connecting controllers and services.
It's also why I don't like the philosophy of unit testing being about only testing a specific class. -- You end up in situations where you convince yourself you have to mock the helper classes it uses, so end up with disconnected pointless tests.
Instead, each test should be testing as much real code as possible.
The best definition for "unit" in unit testing is the test itself. The test is the unit - as in, each test is testing a unit of code and doesn't interfere with other tests or rely on the external environment.
Too many times have I been told a class or a method in a class is a unit. It makes no sense and is the reason for excessive mocking, because they end up trying to isolate that class as an arbitrary unit.
Yeah that's the biggest issue in testing software.
In a perfect world someone would stop and refactor the entire codebase to make every object perfectly unit testable.
In the real world, you have to pick between expanding the system under test (SUT) to include multiple real collaborator objects or creating a violent mess of mocks.
It's not the right thing to do in a perfect world either. I've written quite a lot of software without any dependency injection with 0 unit tests that is fully covered by reliable, more than fast enough integration tests.
Some people think that if I did dependency injection so I could unit test all of that same code, that code would be "better" in some indefinable way. In reality the code would just be longer (dependency injection increases SLOC). All other things being equal - longer = worse.
DI-all-the-things is a dogma hangover from the late 1990s when integration testing and end to end testing tooling was all poor, computers were all slow and people tended to write more of their own algorithms (rather than importing them). DI is now something that should be approached on a case by case basis.
Yep, and I've seen code shipped that passed all its unit tests but the interface between them was subtly wrong so it violently failed to work at all after being shipped. Had to write functional tests to make sure it all worked together.
I'm really interested in this. Most of my systems use DI containers to orchestrate any system-level fakes (e.g. the file system) I might need, but I have minimal mocking or fakes otherwise.
Do you have a blog or article that goes into your design more?
No, I haven't read or written about it I'm afraid.
System level fakes is how I achieve it though. They're often more costly to build up front but they're extremely reusable between projects and you can use them to construct better, more realistic fixtures.
I suppose it is still dependency injection of a sort - but it's dependency injection that is A) stack-agnostic and B) invisible to the app C) connects much better to the outside world (e.g. real databases or filesystems).
I wrote configuration management software. Mocking the filesystem was a disaster. It worked much better to take the whole system that was responsible for modifying the filesystem and just "dependency inject" a real file or directory and do real POSIX operations on it and include the POSIX filesystem as the "system under test".
So as a concrete example if you have a system for editing /etc/{passwd,shadow} on Solaris you can create a tmpfile (even on Linux) that looks like a Solaris /etc/{passwd,shadow} file (a 'fixture') and "DI" that tmpfile into your system and then do operations on that file and check the result. This is testing the "update the passwd file on solaris" system in isolation, so it isn't a proper full integration test of the entire configuration management run, but there's a lot of moving parts under test. It also allows devs to run it on Linux and while you still have to test it in CI on Solaris before shipping, the likelihood of any incompatibility between Solaris and Linux filesystem semantics is pretty low. And it doesn't mess with the actual /etc/{passwd,shadow} file on your CI server. You can worry about coverage of "okay it edits the tmpfile but can it edit the real file" and we had some tests that would really add a fake user and really delete a fake user on the CI boxes as well. In practice I don't think that was necessary since we just required root.
So you're testing everything under the "please create a user" API down to the hardware and not having to insert mocks into your own code or to attempt to mock the entire POSIX filesystem API (I did that at one point much earlier in my career and it still lives on to this day as the biggest mess of a test I've ever constructed -- unfortunately it works and doesn't change much at all and would have taken way too long to rewrite it).
It's also why I don't like the philosophy of unit testing being about only testing a specific class. -- You end up in situations where you convince yourself you have to mock the helper classes it uses, so end up with disconnected pointless tests.
Instead, each test should be testing as much real code as possible.