"I've learned to never ever badmouth a weird architecture in a working and profitable company."

I learned that lesson myself earlier this year. A client had a system that was just so ... wrong, I found myself thinking I'd made a terrible choice of project, and initially marvelled at the fact that the company simply survived.

I stuck it out because it was a short-term engagement and eventually learned that they'd solved an "unsolvable" problem, one that I'd tried myself and seen dozens of other attempts at, all eventual and apparently inevitiable failures. But this place, by ignorance or genius, seemed to make the wrong decision at every step, and did it.

Not to say that I agree with "if it ain't broke, don't fix it" all the time, but when you see something done wrong in a new way, make sure that you extract what you can from it and you might be surprised at what you'll learn.

> they'd solved an "unsolvable" problem

Without giving away any proprietary information, could you elaborate on that? It sounds very interesting.

Ross's talk is significantly more deferential in tone than the Arstechnica writeup may suggest.

I'm glad to hear that. The more I think about this article (and a lot of the comments here) the more uncomfortable I become. There's a difference between "don't be clever", "don't build it yourself", and "don't ever optimize". I find it hilarious that the other high ranking posts today are about Haskell and NoSQL, both of which might be reclassified into the "clever" category in a few years.

While this cleverness probably crimped Etsy at some point in their growth, it also probably saved a lot in hardware costs at a critical time in the business. It's worth nothing that they are still in business, that it was possible to throw hardware at the problem to buy time, and that they now have a solution they are comfortable with. While one can always learn from mistakes, I wouldn't argue too hard with that sort of success.

I think you could argue that using postgres and using logic-heavy stored procedures was positive technical debt, in the sense that it was the approach that one of the founders was familiar with. (Note that I'm definitely not saying that I think doing things this way was an objectively good or even easy path: just that given that particular founder's experience, it probably worked out better than spending the time to learn something totally new would have.)

So was using a monolithic master database, despite all of the pain we have migrating off of it now. The way we work with MySQL now is great at scale, but it does require more work to do many things. It relies on denormalization and presumes the existence of some kind of job server to keep things consistent. If we started out trying to build the architecture we have now, we never would have gotten anywhere.

The python middle layer is kind of a different story. I think it was kind of a blind attempt at scaling to 1000x when we should have been worried about 10x or maybe 100x, and since it wasn't a reaction to a real problem it was closer to being just totally wrong. Rather than allowing us to iterate quickly for a while at our own future expense, building/deploying and dealing with it actually froze new feature development for an embarrassingly long time.

Sounds like the paþ to madneß…

Just because something is successful doesn't mean it's right. I get what you're saying, but it doesn't mean that successful systems are exempt from intelligent discussion about where things went wrong. A lot of poor decisions are propagated because people are building on "successful" technology.