My preference in these situations is always to start with buy, and then try to justify build. Unless it's a core competency to your own business, which it almost always isn't, it makes sense to let someone else do it better.
Their profit margin is usually in their efficiency in doing it better than you, and at the same time you get the benefit of all the features they add for other customers.
Oftentimes the decision to build usually came down to the difficulty in integration of a commercial product, not the cost. In most cases the cost is set to be competitive against building yourself.
Except in the case of Splunk. Their pricing is ridiculous.
> Their profit margin is usually in their efficiency in doing it better than you
This is a very nice way of framing the question.
I think there may be at least two scenarios to consider. For some products, cost is primarily driven by R&D; for others, cost of operating the service (COGS) dominates. Of course it's a continuum, but I suspect there are more services near the ends than the middle.
For an R&D-dominated service, if your needs are nontrivial and there is an off-the-shelf product that is a decent fit, you probably want to go off-the-shelf. A commercial provider can amortize the R&D over a large customer base.
For a COGS-dominated service, the commercial provider may still be more efficient but it is less of a slam dunk. Log management (e.g. Splunk) involves a substantial operational component, because you're dealing with large volumes of data.
Disclaimer: I am the founder of Scalyr, a Splunk competitor. Early on we realized that, even though log management is COGS-heavy, we could achieve huge economies of scale, because query workloads are highly bursty. By carefully managing a central pool of resources shared by many clients, we're able to run much more efficiently than a single-tenant homegrown solution [1]. This is one of the key points that we highlight when having build vs. buy discussions with potential customers.
Seeing how an enterprise software chooses to articulate and differentiate itself in product and sales from the internal perspective is super interesting.
Are there other examples of "we chose to go to market like this (x, y, z reasons) and here's why"?
Thanks for reaching out. It really depends on what you're looking for, and I don't want to hijack this thread with a long discussion, so feel free to reach out at https://scalyr.com/contact. If you're focused on the power of the query language, then Splunk is the undisputed leader, but we are relatively strong as well; see https://app.scalyr.com/help/power-queries. This iteration of our query language was launched last year and was a lot of fun to build -- the team allowed me to write a fair amount of the code. :-)
I was an early Scalyr user circa 2014 or so, and have used Splunk at various gigs before and after. My current company is not a Scalyr customer and that makes me pretty sad - even back then, they were snappy and responsive in ways that Splunk has never matched. Despite (maybe due to?) being substantially cheaper, the service we received from the founders and operators was second to none; for example, I remember that someone even hand-wrote a Clojure EDN log-parser for us at no extra charge, just to make sure we were happy.
Steve says elsewhere in the thread that Splunk has the more-flexible query language, and while that may be true I as a product engineer never noticed the delta myself.
TL;DR I think you'd be doing yourself a favor to try Scalyr, and I have no affiliation other than being a happy former customer.
Hey snisarenko, you could use the Gravwell Community Edition which is free up to 2GB/day. 4GB/day if you participate in the current alpha testing program. https://www.gravwell.io/download
Disclaimer: I'm one of the founders. We built it to be a Splunk alternative and I think it does a great job from a data ingest, scalability, and data querying perspective. We're lacking some of the out-of-the-box capability, but not for long. Kits (our "apps") release is coming this quarter.
It's Pareto principle for me. >80% of our code should be off the shelf. The special sauce is 10% of the code. What should we spend that on? What will make us stand out?
If you can answer that, it's all the justification you need for building it. Anything you build that doesn't differentiate you can be bought by a competitor.
As usual, it depends. I know some friends that have profitable websites because they run on a private server and they handcrafted everything. If they had to use cloud services for serving stuff, host the db, provide search and cache things, it would cost too much and they couldn't make a living out of it.
To me, this shows that the actual service/content they're providing has relatively low value, and so they're "making" money not just on the content but also on the services side; eg, they're a cheaper sysadmin than an outsourced one. And they're also possibly ignoring risk; eg, they don't have the data durability that a cloud provider does, but it Probably Won't Matter (TM).
There's absolutely nothing wrong with this way of making money; the lower the margins, the more watching your costs is crucially important, and the difference between making a living and not even breaking even!
Possibly, it may also be that the cloud providers are offering things they don't need. Maybe that's uptime guarantees, potential scalability or integration with other services.
If you take SaaS, they are skewed towards buy, because their margin is ridiculously high. Better to hit the market now by using stable products than optimizing on spend.
The bigger you are and the more you can think about optimizing spend.
I think it is quite the opposite. Having worked in startups that were VC funded it seems they consider VC money free money and just spend a lot of it in infrastructure without the need to.
I know for a fact our bill was around 15-20k USD per month for running a webapp that could be ran in digital ocean 40 dollars/month.
For a lifestyle business without VC money to burn it makes all the difference.
Oh I've definitely seen that too. My example requires responsible (and smart) leadership. My thought is that optimizing costs engineering hours, which are much more expensive than paying for the service bills.
I agree with the idea, but I also think there are limits.
Let's say you build some kind of product based on some Azure/AWS product. The negative reading of it is: you spend money on innovation and have to pay Azure/AWS for infrastructure. Your product will constantly have to improve to keep up with the competitors, Azure/AWS keeps getting your money. And if your product becomes stable enough, they will make it part of their offering.
Another example would be Apps, with Apple later folding their functionality into the OS and kicking your app off the store.
But judging by my own examples, this migth be more of a platform problem.
Surely that very much depends on whether your product fits within the current or near future offering of Azure/AWS.
The other side of that coin is that you use a build horizontal integration strategy to replace elements of the products you buy from Azure/AWS if (and only if) it can be done in a way that increase your profit/value.
It can work both ways. Moving from buy to build also has elements of de-risking and economies of scale if you do it at the point where there is measurable success/product to market fit.
Building something you can buy too early runs the risk of premature optimisation if not done for the right reasons.
Absolutely. This is largely what I'm betting on with my latest project, that the time you spend spinning up all the "standard stuff" for a SaaS project is duplicated - and thus wasted - effort throughout the industry. Why write another user login system?
Though it sets up an interesting situation for me, where the "special sauce" of _my product_ is the 80% "off the shelf" for my customers.
Another reason I like to start with buy: It makes scope creep a lot harder. You have a specific feature set, and it lets you do certain things, and not others. You may eventually find there are certain things it has that you can't live without, at which point you may still have to build, but that's offset by the fact that you'll probably also discover there are features you only thought you needed.
By contrast, whenever I've been on a project where the company went straight to building their own, it ended up being a quagmire of scope creep, and the development team would end up stuck on an endless treadmill of implementing features that stakeholders vehemently insist on, but never actually use.
Even for dev tooling and libraries. Some of the worst piles of technical debt I've encountered happened when some rockstar decided that they knew better than everyone else, and dived straight into building what they thought would be a better mouse trap. It's not that the in-house option isn't any more or less likely to solve the problem well (no comment), so much as that the in-house option is going to end up being more deeply tangled with the rest of the codebase. So you're stuck with it, because, even if it doesn't turn out to be all it's cracked up to be, migrating away from it may be next to impossible. Homegrown ORMs seem to always turn out this way.
The one exception is when you know that a quick-and-dirty DIY solution will be cheaper and easier, because it needs to do only one specific thing while the off-the-shelf option has to take on a bunch of extra complexity in order to be general enough for everyone's needs. But even there, I'm only likely to trust that argument if it's being made by someone who's been burned by the complexities of the off-the-shelf solution in the past, and who has a proven track record of obeying the KISS principle.
> Except in the case of Splunk. Their pricing is ridiculous.
Their pricing is ridiculous and what is even stranger is that with the ELK stack there is a good and free alternative. Now Splunk is good, no doubt, but I still wonder how they can be that successful with that pricing.
We ran the largest elastic search cluster on the west coast some number of years ago. If you are small enough, elastic search can go a long way. Our data teams and operations folks celebrated the day that that system was turned off. Elastic search does not even hold a candle to the way we leverage splunk at our org, but that could be because we are bigger than some and deal with scale that few others deal with. Splunk costs us a fortune but enables amazing data analysis. It is easily our most expensive service we pay for and worth it. It would be great for our bottom line if it were cheaper, but they can charge what they do because the do it so well.
Splunk is insanely good. To the point that people forget that other tools exist, or that it makes sense to build specialized solutions when needed.
So people start running anything — from dashboards to analytics — from Splunk alone. Their ability to combine and extract data from almost anything in very small amounts of time is unparalleled IMO.
Splunk is incredibly powerful though, and almost all of that power is available at query time. Replicating the functionality in ELK often means indexing changes, and so when you have a question that isn't answered by the index, you'll forgo the answer unless you really really need it. A very simple example is the 'transaction' command in Splunk, which I absolutely could not live without and often surprise myself with the keys I end up using to research a particular topic.
I think people are easy to underestimate the cost of building and maintaining a new piece of software. I mean yeah you can set up an ELK stack (just as an example) on your own hardware / cloud easily enough, but what will it cost in the long run? How many people will spend how much time on it? Or how many people will end up having to spread their attention between the core objective of your business and managing the ELK stack and other services you decided to build / operate yourself?
> My preference in these situations is always to start with buy, and then try to justify build
What if there are significant costs associated with that? Integration, training, friction to move to a new solution later, etc?
When things are commoditized it works OK but for anything complex and strategic, the cost of implementing a potentially bad solution can be absolutely crippling, especially if dependencies on that solution grow rapidly within the organisation.
A story of Netflix adopting StackStorm for autoremediation is a nice illustration of this approach. They began to build a tool, learned enough to be dangerous, discovered StackStorm and used it, experimented with it like with a breadboard, until finally figured out exactly what works - and reimplemented it. All these time learning the system was working and delivering vale.
We self host splunk and it can plow through petabytes of high cardinality data pretty dang fast. If the fields are not indexed and the search is complex, it can take minutes or hours. But usually, I can get live and historic data in a few seconds.
As an example, we have a pipeline of services. I can compute the time spent in each service with multiple levels of percentiles and group that data by high cardinality fields (as in, hundreds of thousands or more values). I just did a search for 4 hours of data across thousands of nodes for half a dozen or so services with multiple eval statements all piped to a timechart doing over a dozen stats operations. Half a billion events. It got done in under a minute.
Splunk charges so much because they are just so dang powerful.
This has not been true in my experience. I run a Splunk server in production and at my data volumes it has been very performant. It's also much easier to setup and maintain than ELK clusters.
In the early days Splunk pricing was exorbitant (we evaluated Splunk 7 years ago and dismissed it), but licensing has changed in recent years and it is now priced by volume ingested (the pricing is transparent and listed on their website now). At low volumes, the pricing is similar to Sumologic, and is pretty accessible now to smaller dev shops. Open-source collectors like fluentd also help to intelligently reduce the ingest volume.
Does the speed matter? How exactly?
I am genuinely curious: at Scalyr we _can_ be very fast but it is a balance with the cost that we want to pass on as price savings. Same with self-hosted Elastic: one can fine-tune it to be fast but minding the cost constraints gets it slower. WDYT?
Yes same here. I actually monitor the throughput of the network interfaces on our forwarder with prometheus/statsd_exporter and if outbound is smaller than inbound it sets off alerts!
This is the core of the whole debate, in my experience:
"When it comes to build versus buy, the frequently repeated but rarely followed wisdom is good advice: if you're a technology company, vendors usually generate significant value if they're outside your company's core competency; within your core competency, they generally slow you down."
Every startup I've ever worked with has had engineers who advocated "build" for tooling that fell both outside the eng team's workflow and our company's core competencies.
I think part of it is that things like marketing automation or customer support software—things that are vital for most teams past a certain scale—don't seem technically "hard" enough to justify the price-tags, particularly to engineers who've been isolated from other parts of the business.
Without fail, every time I've been part of a team building an internal tool that we/another engineering team won't use, it goes poorly. I've spent way too long learning about the ins and outs of email deliverability so that I could fix a bug in a hacky email platform, when we could have just bought a platform that works.
Agreed - problem for me, as an engineer, is that I do like building those things that are outside my core competency so I can gain a wider variety of knowledge and experience.
Definitely not worth it from a business perspective, but has resulted in some interesting toy/hobby projects!
> I think part of it is that things like marketing automation or customer support software, don't seem technically "hard" enough to justify the price-tags
As someone who's built far too many of those because "Buying is just too expensive" ... good god do I never want to do that ever again. Ever. It's so fucking hard. You will shoot yourself in the foot. You will suffer. And 3 years later you still won't have something half as good as the purpose-built SaaS.
If you do that and enjoy it, just spin it off into its own product. Who knows, you might even get more customers with more expensive problems than the original startup.
This only makes sense if you have a significant equity stake in the company you're working for. For the vast majority of engineers, they really don't care about the long-term profitability/success of whatever company they're working for. They just want to get paid, and hopefully grow as a dev in the process.
They'll learn more by building something in-house than by bringing in a saas and just writing glue for a living. If you want your engineers to make the decisions a founder would make, give them a significant equity stake.
It depends. Significant stake helps of course and despite that, I'd prefer solving problems that need solving than spending a bunch of time on problems that don't :)
I've worked at Not-Invented-Here shops and it was annoying that the Powers That Be didn't want to use a COTS solution. As much fun as it would be to reinvent DynamoDB, that's not actually going to help our business.
However, even more annoying is working at a Never-Invented-Here organization. I've been at a place which hired brilliant people, but then didn't let them innovate on pretty much anything. Write a simple TCP listener in Python to read a stream of data and then reply to it? That's crazy talk! We need to write a full Django app and deploy it behind Nginx behind a load balancer so that we can pretend it's highly available!
Don't get me wrong: the off the shelf approach is usually the right thing, but sometimes it just adds complication where it didn't need to be. It's insulting to be told you were hired for your expertise, but then not be allowed to actually use it.
“Write a simple TCP listener in Python to read a stream of data and then reply to it? That's crazy talk!”
This has infected my company quite a bit. People get really nervous as soon as you write something that’s little novel. Writing everything from scratch is bad but you also need room for people to innovate.
It’s shifting the complexity from development to operations ( or “gluing shit” as a sibling has pithily observed.
I’ve found this to be common in places with lower engineering skills. The “engineers” are mortally afraid of building anything novel for fear of failure; the COTS approach can get many clever folks pretty far in their career.
Don't take the wrong message from Joel here though. Make sure it's actually a core business function. Is it something that will actually give you a competitive advantage if you do it better than everyone else?
I think the advice holds. Cryptographic primitives are vanishingly unlikely to be your core business, even if the things built on them are. Therefore, almost nobody should be building them.
Super important exception. Thanks for mentioning it!
Unless you've spent the last 10+ years working on encryption full time, please don't try it yourself.
I used to think I could write crypto. Then I learned more and realized it needs specialized knowledge I don't have the time to acquire. Then I spent some time trying to acquire it and figured I was OK to try again. Then I watched many other more experienced organizations crash and burn. My current position is that I know enough about crypto to know I never want anything to do with authoring it.
I don't think that's an exception. It's just that encryption isn't actually core functionality for most businesses (even some that think it is).
And if it is core functionality, you should have staff qualified to roll their own, and you should all know why you are doing this (i.e. what differentiation you are going after). This really isn't limited to encryption, it's just a hard domain that bumps up against a lot of other domains, so you see more failures than some others.
Basically if you are going to build a startup or new program in a hard domain, you need to have the resources to do it properly, and you need to know what those are before you start. This is where being a team of smart and motivated individuals may just not be enough.
I'd argue that in a startup, at a very early stage, you can use external services even for core functions, if they work well enough.
Look at Intercom: now it's a juggernaut, but it started as an API mashup with a UI in front of it. If you can hit the market and start getting revenue faster you should. Many services have pay as you go and/or startup discounts.
That's your launch platform, then you start working from there.
I think it’s good to buy something if you can use it immediately without much customization. If you have to do a lot of customization it’s not that clear. For example we have bought a system that requires extensive customization and a lot of people are starting to wonder if the total effort of building from scratch that does wha we need and not more would have been less than the never ending customizations of a system that can do way more than we need.
There is also the question of brain drain. The IT department in my company has outsourced a lot of the tech work to the point that there aren’t many people left who are capable of making informed purchasing decisions. They mostly read sales presentations, white papers and Gardner reports but they have no ability to really assess things. This shows in them repeatedly buying expensive systems that never get really used.
I can attest that the sales presentations tend to be overly optimistic and cheery and if the head guy really wants a given solution, from office politics perspective, you might as well just nod your head, point out the issues you think should be addressed just so that you are covered and roll with it. In practical terms, the decision is not up to you.
In one of my previous roles, manager simply said it is happening ( so it did ), spent some time with the system not fully understanding the underlying logic and was somewhat surprised, when it did not work as advertised. IT had to get involved to smooth some edges. The roll out was still painful. But the contract was already signed and the advertised feature shown during presentation was now extra money, so we were forced to use incomplete product. Annoying.
But then, the real answer is that it varies. Everyone needs to do their due diligence.
What I did not notice is that managers see pretty dashboard and fail to consider the use of the rank and file users. This is the one thing I have learned from all this.
Never underestimate the impact of a pretty dashboard.
That is integration and evolution costs on the article. Those tend to be very hard to estimate, and are the ones that most grow out of control. The article really doesn't do them justice by passing through them like if they were like the others.
There is also an undereported gain that is freeing your team to work on higher impact work.
The issue is really about missing requirements... we spent 1,500 dev hours customizing something only to realize we were using the wrong tool for what we wanted... kinda hard to sunk costs that one.
I think they would produce better outcomes if they understood or used the the systems they are buying or at least listened to actual users. I bet healthcare.gov would have gone better if some high level people had actually have tried to sign up for themselves.
As someone who helps software vendors overhaul their messaging to attract and convert buyers, I've noticed the following things about the "build vs buy" dilemma (or, from the vendor's point of view: the "build vs buy" objection):
- More senior buyers (CTOs, VPs) are less likely to pit the vendor against a DIY option. I suspect it's because they've been through too many failed DIY projects, or they have great clarity on what is and isn't a good use of their people's time, or they are better at estimating the (usually huge) amount of effort that would go into DIY.
- As the OP points out, a company with huge engineering resources is more likely to consider the DIY option, because, hey, they have all these engineers that need something to do. So if you're a software vendor and are tired of the "build vs buy" conversation, stop trying to sell to other software vendors or VC-backed tech companies.
- If someone chooses the DIY route, just set a calendar reminder to check in two months from now. Chances are they'll be underwhelmed with their progress, and an out-of-the-box solution might sound pretty good by then. (Don't assume they'd reach out on their own in that scenario--they've probably spoken with dozens of other vendors since your last conversation, and forgot about you.)
===
Separately, the part about vendor management is gold:
> To get the full value from vendors, you have to invest in managing vendors well.
I've spent so much time helping evaluate and implement different vendors (usually marketing platforms and similar), only to find out three months later that the software barely gets used.
On the flip side, if you're a software vendor, don't wait until renewal periods to check in on your customers and make sure they're getting the maximum value from your product.
I'm surprised "pricing transparency" isn't included here. A vender that requires you to call a salesperson and do the sort of negotiation on price that is mentioned here is always going to take advantage of you.
My policy is: no pricing on the website, requirement to call a salesperson to talk price? No buy. I'm even willing to pay more to a company with transparent pricing that doesn't require negotiation. It gives me predictability, rather than have to worry that once they've "locked me in" they're going to jack the price and crank all the gears.
We had the same issue with splunk. We were blowing through our licenses like a champ.
However the thing that managed to reduce our dependency on splunk wasnt ELK or greylog (they are all poor comparisons to splunk in terms of speed or chopping and changing data) it was grafana(with cloudwatch and graphite, later promethius)
What we realised was that the primary way we were consuming splunk was in graphs. Why spend millions of pounds on generating graphs indirectly, when we could make them directly, in near real time!
Not Tangent:
I think like platform choice, it depends on your company. The company that I was with was utterly incompetent, and unable to see a project through to completion. Therefore if we build a logging system inhouse, we'd get 40% of the way there and rebuild it. Leaving two incomplete systems.
Migrating to splunk cloud was done to save costs, and it actually worked. Had we tried to go inhouse, we'd still have expensive local splunk _and_ a shitty splunk replacement used by two team on a legacy critical system with no dev hours.
Article felt more like "not buy", not "build vs buy".
I just spent an afternoon helping someone to get us to run on-prem. Phone calls! Installs! Vendor management! We didn't even setup TLS b/c gov stuff is complicated! Working with vendor software is so painful!
Except they were originally thinking they'd have to build what we've spent years not only building, but building as GPU / data / etc. specialists, and we've already adopted many best practices for stuff like TLS and navigated other gov systems for them. (It can still be way better, but imagine if you had none of that stuff, and then had to support it!). Even ignore how we're special -- GPUs are magical if you use them right -- just that we have basic stuff built in and with on-going support. That's so key and adds up over the years. We do it for our graph investigation niche, but same deal for almost everything.
As software has matured, and I see more and more orgs dealing with the reality that they're not Google nor Stripe with thousands of the best-paid software engineers, I'm convinced articles should be explaining from the default of "not build".
In my less charitable moods, I'm inclined to say that you're unlikely to find a whole lot of software developers saying you should build less software for the same reason I've never met a real estate agent who's really on board with my desire to stick to a more modest living situation.
Besides, building in house is inherently more fun. What would you rather do, write new software, or pore over manuals for software that someone else wrote? For my part, there's a reason I became a programmer and not a tech writer.
Yep! I find that working with management involves planning through outcomes and general resourcing, while working with developers and engineering managers involves planning through the cooler stuff they get to do. ("Instead of reimplmenting some JavaScript UI and Python backend thing for the nth+1 time, we'll zip ahead to the GPU era and focus more time on the data engineering and data science and visual analytics side...")
Folks don't get recognized as heroes for not doing anything, but when they drive the greater story, everyone wins. Understanding the individuals involved in successful software delivery is so important!
There is so much stuff to build that to pass up an opportunity to build something everyone is building is no big deal. As TFA says: focus on the business, build to its core competency, buy the rest if you can (you can't always).
I'm super surprised to see this article not mention the phrase "opportunity cost".
That's what someone pays to save with something like Splunk. Pay cash to make SWEs more effective towards solving the problems that make up the business's value prop. Cash is much easier to get than the opportunity to grow the business with the people already there. That time never comes back.
To me the primary thing that matters is how strategic the software is to your core business. This may sound like just evaluating whether it's your core business or not, but it's a nuance on that which is important.
It's easiest described by an example: we use open source Gitlab extensively. Numerous times it has been proposed that we migrate to $expensive alternative because they all have advanced functionality that we don't get from Gitlab (in many cases, even the enterprise version). Yet we are sticking with Gitlab, and making extensive customisations (mostly using the API) and workflows adapted to it.
Is our core business developing software for CI/CD/Issue tracking/source code management? Absolutely not. But is that critical to us? Absolutely. Because Gitlab is open source and because it has an extensive API, we can eek out efficiencies all over the place that aren't possible with closed source software.
So I think it's really important that people use the right frame of mind when evaluating "core business" or not. You don't have to be in the business of a selling particular type of software for it to be worth doing "in-house". You do have to be good at it though.
When you are looking at paying for an expiring license or SaaS product, it is build versus Rent. And one cost of renting a product is that your vendor's business may change. Perhaps they get bought by Apple and the product is shut down, or maybe they just decide to go in a different direction. Or they raise their prices. Or the service isn't as good as you thought it was going to be. If you have done a lot of integration work (either in software or organizationally) you are either locked in or all that work is wasted.
This is addressed in the article as "risk".
The best way to balance this is to prefer buying/renting Open Source software. That way, if your vendor changes direction, or their service level drops unacceptably, you can choose a new vendor without losing your integrations. Or even switch to a self-supported model.
And any improvements you need, you can contribute to the project so that others can take advantage of them and share the maintenance burden.
Personally, I (as a software engineer who loves to write software) would advocate buy anywhere and everywhere you can. Even your core business. In my personal experience it's rarely code that's giving you an advantage, it's more likely to be your data, your connections, your efficiency, etc.
Also, try to use the product as off-the-shelf as possible. Little or no customisation. If people in your business are unable to work with a product without customisation consider firing them and replacing them with people who can before you get involved in customisations.
For me the dominating factor here is human resource pool. For whatever we build there are no experts on the system except the ones who already work for us. So aside from all the build, maintenance, etc. costs associated with build, I also can't find any employees on the market who are experts or even competent in the software.
One thing overlooked here (or not explicitly called out) is talent pool. My last boss didn't want to use anything that he couldn't plug into Dice Or Similar and get dozens of hits for.
If it's not part of your core business/core competency, and you're just reinventing the wheel so you don't have to pay a vendor, you have to weigh against the cost of the off-the-shelf solution NOT just the cost of developing it yourself, but developing it, maintaining it, documenting it, replacing your "irreplaceable engineer" when s/he quits, etc.
Sometimes, you don’t get a choice. In most large enterprises, it is almost a given that you’re going to buy instead of build, and that thing you bought better come with enterprise support.
I think it's all about focus. There is a limit to how many goals/projects you can focus on at once. Every new thing you build distracts your focus. You should minimize what you build in-house to just the things that make you unique. I like Unsplash' blogpost about this: https://medium.com/unsplash/scaling-unsplash-with-a-small-te...
One thing that's commonly overlooked is the risk/vendor lock in associated with building things in-house. Sure external vendors sometimes raise prices or shutdown their product. Those events are pretty rare though and you can typically move to a competing solution. The real risk is building in-house and having the people that create the in-house solution leaving.
I guess the real question is how to correctly distinguish between a commodity vs a potential lock-in. For example, if you stick EC, S3 and SES, it feels more like a commodity. However, with more services in the basket you will probably end up in a lock-in situation and your company will hardly survive if e.g. your AWS account gets suspended.
"A company that does this extraordinarily well is Amazon, who issue their vendors quarterly report cards grading their performance against the contract and expectations. Getting great results from vendors requires managing them. If you neglect them and get bad results, that's on you."
- Gives JIRA a report card on them as a vendor to your business.
- Is largely ignored because my business is not Amazon scale with all the power that holds.
tldr; this only really works if you're one of the whales for a vendor.
When your core existence depends on being the only people able to do a thing, being able to do it with off-the-shelf product begs questions which go to the axium of you being the only people able to do it.
If you understand the role, this doesn't have to be a problem. Hence registry/registrar/delegate separation: the registry is not the delegate. the registrar is not the delegate, and has to meet the demands of systems boundaries implemented by the registry. The delegate has no software limitation and no need to own software because its a role divorced from software. Anything can be out or in source.
Or the CA/RA functional divide in a PKI. Better not to be an amateur implementing crypto, buy off the shelf because your role is to adjudicate what happens, not write systems.
Build, buy, and engage if it's part of a larger project that involves any innovation. Engagement falls under "vendor management," but that seems like a very reduced description.
Their profit margin is usually in their efficiency in doing it better than you, and at the same time you get the benefit of all the features they add for other customers.
Oftentimes the decision to build usually came down to the difficulty in integration of a commercial product, not the cost. In most cases the cost is set to be competitive against building yourself.
Except in the case of Splunk. Their pricing is ridiculous.