"This'll take an afternoon" - three weeks later......
Programmers are notorious for this.
BUT even apart from this problem ... you absolutely should use every dependency you can that will save you time.
Try to write less code not more. When you write code you write bugs, add complexity, add scope increase need for testing, increase the cognitive load required to comprehend the software, introduce the need for documentation..... there's a vast array of reason to use existing code even if you truly could estimate it and build it in an afternoon.
You also assume that you understand all the edge cases and fickle aspects of the dependency, all the weird ins and outs that the dependency author probably spent much resources understanding, fixing and bug hunting.
There's a hard fact that proves the above poster to be wrong..... how many dependencies took only an afternoon of time in total to write? Hard to say (maybe look at the github commit history) but I'd guess almost none. It didn't take the dependency author an afternoon, so why will it take you an afternoon?
Even worse .... you just lost an afternoon coding features for your core application.
Multiply this by every dependency that "you could build in an afternoon" and you'll be in Duke Nukem Forever territory.
I'd advise doing the opposite of this articles suggestion.
Find a dependency that will save you an afternoon? Grab it.
- Dependencies break over time. They have a nonzero maintenance cost.
- They impose API boundaries on you that may not fit your existing data structures
- It's harder to change underlying bugs
- They might introduce security issues
Sure, use dependencies. But there's a reasonable position between "never write any code" and "never take on dependencies". Of which NPM is one of the only ecosystems being at one extreme.
This is pretty much spot on. Except you also missed the main cost, which is the insane amount of time it takes to learn the 85 dependencies on your project to an extent that you actually understand what your code is doing.
Every single project I go into seems to have a smorgasboard of dependencies, then when I take the time to investigate one of them I find out it's being used incorrectly by at least 50% of the team because they don't even understand how they work at the most basic level. Which is pretty understandable because by the time anyone gets through understanding 10 of the 85, they've probably been kicked off the team for not actually building anything.
People love to say rubbish like "write less code!", as if LoC is the only metric that matters (weren't we past that thought process by the 90s?). Which goes a long way to explaining all the fucking terrible codebases I have to work with where it's impossible to accomplish anything without reading documentation for 8 hours, when it would take 20 minutes to just read even a semi-readable piece of code that implements whatever requirements you need from the dependency.
On a C code project for a large Fortune 100 company a half dozen years ago, I encountered a pesky header include that made no sense. And that header was part of a patch that I really did not want to pick up, so I started digging into it.
Turns out that they had some constant in the code, and the developer just did a grep for that value in the source tree, and that constant already existed in an existing header file, so they just included it.
And that CONSTANT_VALUE_STRING had nothing to do with the technology that the C source was addressing. So some lazy slacker pulled in a random header file that contained the proper constant value for an unrelated technology.
The dependency on that was pure lunacy on so many levels.
And that was an internal dependency, not an external library.
So the lesson here? Not all dangerous dependencies are external.
I could say all of the same things about the in house tools that the “architect” wrote three jobs ago - including the bespoke ORM, object mapper, and logging framework.
Or two jobs ago where two developers who had worked at the company for 10 and 15 years respectively were maintaining a bespoke 15 year old EHR written in PowerBuilder and depended on SQL Server 2003 - in 2016.
Every company thinks they are their own special snowflake where cross cutting concerns can’t be handled by a third party.
Funny you should bring up these as examples. I've made (1) and (3) using an existing object mapper. They were made out of limitations with JPA/Hibernate and for higher-level functionality in logging. The ORM was never sent to prod. The logging events were gold. Specific events were 'major' and filtering on a user ID in a narrow timespan could show the expected/unexpected events for the traces as a sequence diagram through all the microservice layers. Clicking on an event then searched Loggly for all logs for trace-id at time. It got to the point that non-techs were answering customer issues with it and we hardly had to check/wait for Loggly to answer.
Everyone who's used NPM in production for a not-insignificant amount of time has realized just how bad nodejs dependency hell can be. Unfortunately, webdev-du-jour has decided pulling in a hundred npm packages is better than writing a few hundred lines of code.
I keep hoping things like [1] are a joke but I'm starting to suspect they're not.
I'm sorry that my framework and bundler are using so many packages. Lemme just quickly install Android Studio and download a few gigabyte to develop and build my application. Ah yikes I'm on a different version, need to redownload now.
At least Android Studio doesn't break when you try to deploy it a few months down the line (with package lock), with the exact same version, because a dependency of a dependency of a dependency made an unreviewed and untested "security fix" that caused a regression.
And when you run into a bug or design problem in a dependency of a dependency of a dependency?
It often takes less time to write some code than to understand someone else's code.
Most programmers I've worked with get lost easily when jumping through layers of other people's code. I certainly do.
Solid, well tested dependencies that solve hard problems are worthwhile. But dependencies have a cost in debuggability and maintenance, so it's worth using them with care. And often, they aren't worth the time, when compared to writing a dozen lines of code.
I get easily lost jumping through layers of code written by corporate developers in an afternoon. I generally don't have problems jumping through layers of popular, well documented and single-purpose third-party libraries.
While I agree that if you think it'll just take an afternoon, for the sake of this article it had better!
But conceding that charitable assumption to the article, I agree with its basic premise: dependencies cost a lot of time in diffuse, non-codey ways.
There are AAA dependencies you pull into every project, but most other dependencies require a good degree of due diligence, evaluation, risk, and their own long-term maintainance.
Its not that it always tips the scales all the way to 'roll your own', but
I think the cost of new dependencies is underrated.
But that analysis is part of the design process on the front end. You don't just 'take' libraries or utilities without evaluating them. And you don't just write bespoke libraries without thinking about the APIs.
So do your upfront work, by all means. It isn't an all or nothing decision.
> you absolutely should use every dependency you can that will save you time
> Find a dependency that will save you an afternoon? Grab it.
Agree. The point of the article, though, is that dependencies are often saving much less time than they promise - so much less that it's better to avoid them.
> "This'll take an afternoon" - three weeks later......
> Programmers are notorious for this.
From my experience with these personal failings, the problem usually comes from the question being phrased in the context like, "before you begin working on this, how long do you think this will this take you to complete?". If there's no opportunity to scope, with requires not insignificant work towards the solution, the estimates will always be wrong. If I understand the actual scope of the problem, which means have the architecture mostly worked out, and have a bit of experience (and luck), my estimates can be pretty close, usually eaten up by that oh-so-seductive feature creep that ruins my work file balance.
I recently read through the (free online) 'book' on Basecamp's 'shape up' methodology; I thought the 'hill chart' describes this really well - the work needs to be in progress going up the hill discovering what it's all about, before you get to the top and can accurately assess how much 'real work' (!) there is to do, and then it's all downhill from there.
Exactly. I'm not reinventing the wheel. I may write some convenience wrapping around Spring Security, for instance, but why would I rewrite auth-z when it's a solved problem?
Somehow here we assume a good programmer routinely makes errors in time estimation by an order of magnitude, yet conveniently forget cases when, say, a non-trivial GPL library is embedded as a dependency into a project, and customer is asking for code, and legal team runs with hairs on fire because company didn't plan to release the code...
But that is a completely different topic. Licensing is an issue, yes, but that is part of the upfront decision process.
In this day an age, this many years into open source licensing, if your team is not on top of that from day one, they have failed as a team.
I worked at Bell Labs from the mid 1980s to 2000, and by early 1990s (1992? 1993?) they already had a full internal team dedicated to open source licensing issues, including training and consulting. That was 27 or 28 years ago. Before some of the developers on this thread were even born.
> you absolutely should use every dependency you can that will save you time.
Absolutely. As long as it does save you that time over the foreseeable lifetime of the project. Or you are deliberately incurring a technical debt because of some deadline.
On the other hand, saving an afternoon (or even a week), over the next two weeks means very little.
Essentially, it'll take you an afternoon to write and then weeks of work properly fixing the bugs and handling the edge cases. Potentially and probably, while you're trying to do something else.
"This'll take an afternoon" - three weeks later......
Programmers are notorious for this.
BUT even apart from this problem ... you absolutely should use every dependency you can that will save you time.
Try to write less code not more. When you write code you write bugs, add complexity, add scope increase need for testing, increase the cognitive load required to comprehend the software, introduce the need for documentation..... there's a vast array of reason to use existing code even if you truly could estimate it and build it in an afternoon.
You also assume that you understand all the edge cases and fickle aspects of the dependency, all the weird ins and outs that the dependency author probably spent much resources understanding, fixing and bug hunting.
There's a hard fact that proves the above poster to be wrong..... how many dependencies took only an afternoon of time in total to write? Hard to say (maybe look at the github commit history) but I'd guess almost none. It didn't take the dependency author an afternoon, so why will it take you an afternoon?
Even worse .... you just lost an afternoon coding features for your core application.
Multiply this by every dependency that "you could build in an afternoon" and you'll be in Duke Nukem Forever territory.
I'd advise doing the opposite of this articles suggestion.
Find a dependency that will save you an afternoon? Grab it.