There's some neat tools here, but it also feels like a bit of NIH. It would be interesting to hear why these are better than, say, Gulp, Yeoman, Hubot, Casper.js, etc.
Absolutely. It's a fine line. We haven't always made the right call for sure.
I think Metalsmith and Khaos are pretty good examples of why we feel the need to build our own.
The crux of it is that the alternatives (Yeoman and Jekyll) aren't designed to be augmented without opinion. They are designed as "solves all your needs tools and gives you the preferred workflow".
Metalsmith doesn't claim to solve all your needs. It actually doesn't solve any of your needs out of the box, unless you need a really complex way to copy files from a directory to another without doing anything else to them. It defines an extremely simple core—an API that extendability can be based around—and then hands over all of the logic to plugins. Which means it's pretty much never going to get in your way.
Similarly with Khaos—which we haven't really perfected yet—we're looking for an extremely simple core API design that people can augment to solve their scaffolding needs. We want it ourselves to scaffold integrations for Segment, to scaffold new Node and Go projects, we're even thinking about scaffolding Segment-branded PDFs.
Yeoman doesn't think that way, evidenced by this paragraph from their landing page:
> Through our official Generators, we promote the "Yeoman workflow". This workflow is a robust and opinionated client-side stack, comprising tools and frameworks that can help developers quickly build beautiful web applications. We take care of providing everything needed to get started without any of the normal headaches associated with a manual setup.
Khaos doesn't prescribe any workflows or client-side stacks, it just lets you scaffold things. What you want to scaffold is up to you, as it should be. You can use it for eBooks for all we care :)
Khaos also happens to be built on Metalsmith. It's just a series of plugins that run before the files are written to an output directory. That wouldn't have been possible with Jekyll, we'd have had to roll our own again inside Khaos or do some incredibly hacky shit to get it to work.
---
All that said, some of our projects suffer from NIH. And when they do it's usually clear over time since we get sick of dealing with them and we use something that has more widespread support behind it. But for others, we're super glad we've rolled our own because we get tired of other people's opinionated software :)
There are definitely lots of existing pieces of software that we like to use though. I've been thinking recently it would be cool for companies to have a Chrome Extension that the engineering team could use that specifically marked GitHub repositories as "liked/disliked" in terms of adhering to the companies design ethos. That way we could keep track a lot easier of the stuff we like to default to, so that we don't have to roll our own all the time.