Sometimes, companies like Parse make me think having quality backend engineers is unnecessary for a startup. AWS made it cheaper to run scalable applications, Heroku made dev ops ridiculously easy, and Parse may eliminate the need for backend engineers. The outcry against Parse amongst some of friends who are backend developers is reminiscent of factory workers who feared the advent of machines.
I mean for prototypes, proof of concepts, and maybe 2 man fly by night 'startups' trying to get initially backing to hire backend engineers, great idea. I can even see certain 'startups' may be profitable and sustainable and just never need to invest in 'custom' backend code.
But as a general rule of thumb, in no way should software startups that base their existence on client server software possibly think its a good idea to have no one in house who actually knows how to do server software.
PaaS and services like Parse are the next normal. The work you're describing is called manual labor and where possible (aka the 99% use-case) it will be replaced by better technology.
I remember the same claims being made in the 1980s and 1990s about technologies like HyperCard, PowerBuilder and Visual Basic.
There was all this hype about how they were going to render C, C++, Pascal and COBOL programmers obsolete, because they'd let non-technical folks build production-grade software systems with ease in record time.
Of course, what ended up happening is that this didn't hold true at all. Most apps developed by non-programmers were quite limited and terrible. Anyone stuck with such apps often had to bring in actual developers anyway. Unfortunately, the developers were now stuck using rather limited technologies that often took longer to develop with than if C or C++, for example, had just been used in the first place.
Any serious software system will require lots of custom code, and this only comes about through the efforts of actual software developers.
I think PaaS's are a pretty natural next-step on the evolution of hosting which is a lot more specific and smaller in scope than "programming" - the stuff they abtract away from you is for the most part just a timesink and chore that has between negative (salaries & exploits & downtime) and limited (bang for your buck, ram, disk io) benefit to do yourself.
Parse takes that model another step forward and just addresses the most common use case - CRUD applications that an experienced developer could create in their sleep but haven't we done that enough times already?
There will still be occasion when your own server/s are the best or only option but fewer than today's developers want to admit.
Obviously, startups need to scale at some point, and to do that they need experienced backend engineers. However, when a startup is still searching for product-market fit and doesn't have an influx of users, it doesn't make sense to spend a lot of time and effort attempting to build a scalable backend that probably won't scale anyways. With cloud code and now integration with Express, there are almost no performance tradeoffs to using Parse for most early stage startups.
Its just not that hard to do a 'serviceable' back-end behind a simple rest API. I mean how many hours is that really going to take you, especially when you consider in the case of parse users they already know what data entities their application uses? A REST API for a dozen or so already defined entities, maybe a weeks worth of work? And then look at what it buys you, actual control over your data, ability to move from parse and pursue cost cutting alternatives, ability to do in depth analysis of your data...
I absolutely think Parse has great value, but when the statement is made that in general it might replace back end engineering for startups... its just wow man. This is precisely why a person who understands the server side of a client server application should in general always be involved if the business is based around client server applications.
1 week is eternity for a startup. A custom backend not only adds time to the initial build of the product, but it makes iteration increasigly complex. What happens if you restructure the product and kill half your features? Rebuild the backend yet again?
The only thing that matters for a young startup is finding product-market. Parse makes that process more efficient than ever.
A person who understands backend engineering would understand the value that Parse provides.
Even a mediocre developer should be able to put together a workable prototype of a simple web app and/or API well within a single week, including setting up web servers, databases, and other basic infrastructure.
Experienced developers are often much faster and more efficient. This is especially true if they're using a language like Python that is quite suited to prototyping, and already offers so much pre-made functionality in the form of libraries and frameworks.
Many of us have learned to be very skeptical of these services that claim to allow software systems to be build rapidly. This often only holds true in a very, very small set of cases. Otherwise, the moment you try to do something unique or unusual, you'll hit one roadblock after another. It doesn't take long before the supposed "time-saving" or "effort-saving" service has wasted more time and effort than it would have taken to develop everything from scratch.
I just had this discussion with my friend today. While it is true that building a simple simple api from scratch is pretty quick, the hack-it-super-quickly api gets really hairy really quickly. Later you have to spend a lot of time refactoring to make the API scalable and easy to add endpoints and functionality. And usually you realize you have to refactor when the poop starts hitting the fan and the hacked together solution isn't holding its weight anymore.
That assumes three things. #1. Product category that fits Parse (obviously if you are dependent on machine learning built on postgresql, might not be a great fit). #2. A start-up that is comfortable with that level of dependency and lack of internal knowledge. #3. A startup who considers the bulk of their value to be in interface or first-move network effect advantage (else, company Z will clone their app using parse in a few weeks after all the hard learning is done).
EDIT: For the record, I really like Parse, and am a paying customer for two CRUD mainly products.
As a designer who builds out a lot of early prototypes for testing, Parse is really amazing. Previously I'd do some localStorage hacks to store simple data for prototype demos but since it was just on one machine it was really limited. Now I use Parse for all mock backends and can "mock" entire user authentication flows, objects being created/edited/saved, first-use state (no objects), secondary use state, error and success messages, the whole thing. I put "mock" in quotes because my prototypes are now essentially MVPs with real data being stored and accessed. It's pretty incredible.
There is an outcry against Parse? Seriously? As a back-end engineer, I always thought it was a clever niche product and I was happy (but a bit fearful) when they got acquired by facebook. Nice to see they are still trying to innovate.
The factory workers feared machines because the machines could precisely replicate their efforts. If backend engineering efforts were replaceable with a one-size-fits-all API like Parse, I already would be out of work (instead, everyone is hiring). Parse looks great for a client-driven project that essentially needs storage and some transformation, but I'm no more fearful of it than a good designer is fearful of Bootstrap.
It seems that, one by one, the MBaaS players are realizing that in order to be "complete" solutions they need to offer the ability to execute arbitrary code on the server. You can see examples of this for Kinvey [1], Azure Mobile Services [2] and Cloudmine [3].
Of course, they all do is slightly differently, and there is generally more sandboxing than what you would expect from a traditional PaaS like Heroku. The danger is that by being more complete, they lose focus on their original mission: to bring powerful app-centric functionality to traditional client-side developers.
I have a solution: drop the normal VM-based servers and instead use WebRTC or some other peer technology so the client (browsers or whatever) talk to eachother directly.
Unworkable because of security you say? Add encryption.
I actually think that not too many years down the line this type of thing will be common.
Can someone translate this for me? I'm not sure what it would mean to have 'Parse Hosting', 'Cloud Code', etc... I've only recently started to learn nodejs and didn't have any trouble pushing my tutorial made app to Heroku. Is this just an alternative to Heroku that hosts your app?
It sounds like you've nailed it. This has fewer features, but if you use Parse's data storage (which is their primary product) you probably have improved performance since your data and app are presumably nearby.
It think it's just an easier way for Parse users to do server-side rendering of their single-pag apps. To be honest, I don't even think that's right.
They're coming out with all these interested disparate services, they need to provide a coherent story for how developers can use them together. Maybe a sample app or two.
I believe that within a few years most of the JavaScript programmers out there who are afraid of VMs are going to realize that it really isn't that hard to deploy Node (or anything else for that matter) to Linode, Rackspace, Digital Ocean or AWS.
Until then, they will continue to pay crazy markups to Heroku or live with limitations like with the new Parse service that doesn't let you install any Node packages (there are 31000 and counting now in the npm registry https://npmjs.org/).
Linode even has a StackScript that sets up Node for you. And AWS has something similar with a Node AMI (in the marketplace at least). Or you could just copy paste like 10 lines of code from the Node.js wiki. Or find a script online to do it.
I would honestly like to know what is it that, say for example Parse, has done with their VM configuration that is so much more sophisticated than what people could get from following a few wiki pages on the web, or just getting a good bash script. Because I doubt that there are a lot of extra security or performance features or anything like that.
Configuring and running an Ubuntu server is not that hard.
But maybe it is. Maybe I am just really ignorant and there is some kind of complex configuration or tuning or firewall or daily task that Parse or some other Node.js hosting company is running on their servers that is completely beyond me or that I would never be able to find out about from the web. But I think that for 95% of people nothing complicated is really required.
I really appreciate any hints about Linux server management to clue me in if there are a lot of commands that I should be running on my servers that I am not aware of.
What I know is that my Ubuntu Node servers get running and stay up without me entering a lot of commands or doing a lot of monitoring or anything else.
If you go with AWS, they have a built in interface for the firewall in the browser. You don't even have to use a command line. Although honestly in Ubuntu how hard is sudo ufw enable; sudo ufw allow [port]
It's not hard to install nodejs, and then you need forever or w/e to keep it running, you need that to start on boot, you need pingdom or whatever to monitor it, you need your haproxy or nginx for load balancing, you need to decide what to log and how to store those files beyond megabytes, maybe you need ssl, you need a way to get the next version running, and you need to be able to fix it when stuff goes wrong.
All of those things are just distractions from whatever your project really is. Are you building software or are you dicking around with servers?
99% of Node applications do not need any load balancing. 1% are Twitter or something. And even a lot of the 1% would do it at the application layer using multiple VMs.
And I bet that most of the Heroku apps on Node that use multiple dynos really only need multiple dynos because Heroku's dyno isn't giving them realistic resources on the dynos, i.e. less than one small AWS or Linode or whatever.
Forever will not keep your Node application running. If you Node application crashes, forever will let it crash, and then run it again, and if it crashes again, it will run it again, and it will crash again.
There used to be basic gotchas in HTTP/Express that made it really easy to have an exception thrown that would take down a Node web server. I am not sure those are already there, but anyway I do what every Node expert advises against and include and uncaughtException handler in my Node web servers. That keeps them running. If I take it out, the effect is that they go down briefly, and forever restarts them. So honestly I see no benefit in most cases to using forever except to make it a little bit harder to figure out how launch your application.
And the thing with automatically restarting when a server reboots, honestly my servers very rarely reboot. If they are going to reboot then the VM (VPS) provider gives me warning and lets me control it. So for 99% of applications out there, an upstart or whatever to restart your Node application really isn't that important.
Contrary to popular belief, you do not need to install nginx in front of every Node application.
I will give you the SSL one, that could easily take a good UI developer 2 or 3 days to find the right instructions on Google.
You are correct, with enough experience it will suck a smaller amount of time and attention away from your actual project.
Unrelated to the core argument, I think much of what you just posted is bad advice - forever will restart your app above and beyond uncaughtException, nginx or haproxy let you add redundancy on independent processes if not machines, and restarting your site manually on reboot is ludicrous.
2 years ago I taught myself how to code and design. Figuring out how to set up my environment alone almost killed me. There were just so many cryptic commands that had to be done that weren't exactly english. Then I found the Rails 1 click installer, and it literally did just that. Next I saw the beauty of git push heroku master. Now my page was on the web. It was amazing. To seasoned programmers it might not seem that difficult, but to those who are entering this new world, a few steps and a wiki can start to get confusing, especially if something goes wrong or the person skips a step. Overall the environments for code still have bad UX and I can't wait till someone makes it as easy as the 1 click experience I had.
Ok but what I'm saying is that the UIs for the VPSs are really simple and some of them actually have a way to select a VM with Node already installed or automatically run a script that installs it. Its actually easier than Heroku.
Also if you just had a good bash script from somewhere, that would be easier than Heroku too.
Just basically saying that the right script or server image is all you need rather than paying twice or 10x as much money for something you can't control.
Sure it's not hard to spin up a Node process on a VM, but the abstractions Heroku provides are pretty nice: "git push" to deploy, Procfile to define process types, a simple command line tool for tailing logs, scaling processes, set environment variables to configure, etc.
Plus you need to know a bunch of stuff about sysadmining to properly secure the server, stay on top of security patches, etc.
Setting up and running an Ubuntu instance somewhere isn't hard, but sysadmining is. I think people who pay for Heroku (as I have in the past) are throwing money at a problem. Even suggesting blindly to use "sudo ufw allow [port]" kind of proves this - what ports do they allow? I allowed 80 because web, but now I can't SSH back in to my box. Also, updates? Security patches? Heaven forbid my node app gets popular and now I have to scale.
In other words, it's like the game of Othello - a minute to learn, a lifetime to master. Some people just want to develop.
Do you really think that Parse is doing all of the security updates that come through Ubuntu or CentOS? Is Heroku doing all of them? I doubt it.
And if someone needs to do a security update on Ubuntu, its a one liner.
99% of the Node apps out there will not need more than one VM or any complicated scaling. Or they can scale horizontally at the application level using more VMs, or vertically by just upgrading their VM to have more
RAM/VCPUs.
From Heroku's site: "New systems are deployed with the latest updates, security fixes, and Heroku configurations and existing systems are decommissioned as customers are migrated to the new instances. This process allows Heroku to keep the environment up-to-date. Since customer applications run in isolated environments, they are unaffected by these core system updates."
Which means they do not apply security updates. They decommission servers if they think its necessary. How many times has Heroku actually done this? I am sure it is not with every security update.
Do you really think that Parse is doing all of the security updates that come through Ubuntu or CentOS?
Yes. Yes I do.
This whole line of argument is one I typically hear from folks who've never had to deal with a Sev 1 incident in the middle of the night. If your argument holds, sysadmins and DevOps teams are essentially pointless.
Moreover, even a developer confident handling both maintenance and incident response should value his or her time more than zero. Given that, presumably there is a price point at which it makes sense to simply pay a platform provider or hire dedicated DevOps. There may be varying opinions as to what that price point is, but it does exist if the developer has the cash.
What on earth are you talking about?
Maybe you are kidding.
But if you're not kidding.. are you saying that those VPS providers like AWS, Digital Ocean and Linode are not 'event-driven asynchronous'? Wat?? VMs run whatever code you want, asynchronous or synchronous or whatever.
I think you must be messing with me. Especially since you put 'webscale' on the end of that.
Parse doesn't provide a node.js environment. It's a custom V8 runtime environment, with commonjs and node-like modules. It's really neat, but it will be interesting to see how they maintain/extend a "node like" environment that's not really node. They run the risk of confusing developers who expect to do anything they can do in node.
Wow, it's interesting to see how these data in the cloud services are backing into custom code, instead of starting with custom code hosting and backing into more of a PAAS approach.
