And so ZEIT, my favorite serverless provider, keeps getting better. Highlights:
- "sub-second cold boot (full round trip) for most workloads"
- HTTP/2.0 and websocket support
- Tune CPU and memory usage, which means even smoother scaling
And all that for any service you can fit in a Docker container - which is also how you get near-perfect dev/prod parity, an often overlooked issue with other serverless deployment techniques.
On top of all that, ZEIT has one of the best developer experiences out there today. Highly recommend trying it out.
And for the perpetual serverless haters out there: this is not a product for you, FANG developer. I think people underestimate how well serverless fits for the majority of web applications in the world today. Most small-medium businesses would be better off exploring serverless than standing up their own k8s cluster to run effectively one or two websites.
I'm not a "serverless hater", but every company I've ever worked with had backend processes that were not tied to HTTP requests. I still keep actual servers around because the HTTP gateway is not the pain point. It's long-running processes, message systems, stream processing, and reporting.
That said, I look forward to the company (or side project) where "serverless" can save me from also assuming the "devops" role.
At my last gig, we were using Firebase, Google's acquired-and-increasingly-integrated serverless solution. It was straightforward to have custom GCP instances that integrated with and extended our regular serverless workflows. In that scenario, it meant the compute instances tended to be extremely simple, as they were essentially just glorified event handlers.
Interestingly, as Firebase evolved during our use, nearly all of our external-instance use cases were obsoleted by more powerful native serverless support, esp. around functions.
All of which is the best of both worlds for serverless: an easy escape hatch to custom instances, and an ever-decreasing need for that escape hatch.
Hello, I'm building a serverless platform. Could you please expand your "It's long-running processes, message systems, stream processing, and reporting" bit?
Not parent, but I have the same question; I worked in adtech and video analytics before, now with social media. It's usually a mix of some REST APIs, which already are very easy to scale and manage without using serverless, with long-running backend processes, such as:
* video encoding;
* ETL processes;
* other analytical workloads;
* long-running websocket connections with Twitter/Facebook/etc APIs.
From my perspective, serverless solves the "boring" part of making REST APIs easier to manage, which were already very easy to manage.
How would serverless be applies to, say, a python script that streams Twitter tweets using websockets?
You would probably use something like a queue[0] that takes in data from the websocket and dishes it out to lambda functions. You might also use something like Kinesis[1] or other alternatives.
Yes of course, or I could send it into Kafka instead (which makes more sense to me). The point is, how would a serverless process looks like which doesn’t have a REST API and does this long term polling of websockets?
It depends what you're doing with that stream, most basically you would create a nano/micro EC2 instance that will just trigger Lambda events on every new tweet. Or you could create some more intricate script that does a lot of pre-processing and then stores it in RDS or S3, and with each new update to either of those sources kick off a Lambda.
Unless the API can stream directly into one of those sources you'd probably need a long-running process, perhaps running on a CaaS like AWS Fargate.
I guess you could argue where to draw the "serverless" line, at functions or containers, but Zeit is calling this container service "serverless" so I think Fargate would fall into the same category. I think it would make sense for Zeit to eventually support long-running containers too (looks like the current max is 30 minutes, I'm not sure how they chose that number)
Serverless is fantastic for ETL and data analysis, especially for workloads that vary in scale (eg cronjobs). Feed data in, get data out with scaling as needed.
but how do you feed data in? Usually, it's some other service on one of the big 3 cloud providers. I'm using google for my projects these days so it's a mix of Google PubSub and Dataflow.
I think this is the issue/risk with serverless. You either get locked into one of the big 3, or you end up doing all of the ops work to run your own stateful systems. As some of the people above you said, managing and scaling the stateless HTTP components is not the hard/expensive part of the job.
Can't you use a queue service that's essentially just managed kafka/activemq/other-standard-system? I mean sure if you wanted to move off the cloud vendor you'd have to run your queues yourself, but if you're programming to the API of well-known open-source queue system then you're never going to be very locked into a particular vendor.
The short answer is yes, you can do that, but it starts to get nuanced rather quickly. The context of this is a desire to go “serverless” and that solutions like this only give you serverless for the relatively easy parts of your stack. If your goal is to go “serverless” I take that to mean a few things listed below.
1) you don’t have to manage infrastructure
2) you don’t have to think about infrastructure (what size cluster do i need to buy?)
3) you pay for what you use at a granular level. (GB stored, queries made, function invocations, etc)
4) scale to zero (when not in use, you don’t pay for much of anything)
Most things don’t hit all of these points, but typical managed services hit very few of these points. Sure, I can use a managed MySQL, but it only satisfies 1 of the 4 points.
How does one get locked in when it’s a simple function in X language? Seriously, serverless is just an endpoint they provide. You write the code and they handle everything else.
Because the function is the stateless easy part. To make any non trivial system, in a serverless way, you have to use their proprietary stateful systems. IN my case, google pubsub, google data flow, google datastore, Spanner, etc. that’s where the lock in happens.
Right, because serverless is actually just a cover for "de-commoditizing" the cloud services that companies like AWS built to commoditize datacenters. You hit the nail on the head. It's not completely useless to help less technical people solve the problems that folks like you and I consider "the easy part" and so people will find a use for it.
But the primary utility of serverless is an attempt at solving Amazon's problem of being commoditized by containers.
I’d say something more nuanced. Serverless is increasing commoditization of one layer of the stack at the cost of de-commoditizing a high layer of the stack. This is what makes it a hard decision to grapple with. You’re getting very real benefits from it, and potentially paying a very real cost sometime down the road when being locked into the propietary system bites you.
I think all that is still possible in Serverless. I'm not a serverless architect or anything, but that's typically handled by various serverless queues and related event systems.
Break your operation into a series of discreet tasks. For 99% of use cases, if you have an discreet task that takes 5+ minutes, there's a problem. In most cases, it can be split up.
I know of 1 large scale web application that is 100% driven by serverless technologies.
The user experience (as an end user using the website) is pretty terrible IMO. It often takes multiple seconds for various areas of the site to load (bound by the network). It's also super Javascript heavy and just doesn't feel good even on a fast desktop workstation.
Authentication is also a nightmare from a UX point of view. Every time I access the site it makes me click through an intermediate auth0 login screen.
I really hope this doesn't become a common trend to run web apps like this. It would make the web nearly unusable.
I'm not a "serverless" hater either. I want technology to move forward to make my life as a developer better. I don't care what tech it is, but right now I just don't get that impression from serverless set ups (both from the developer and end user POV).
Confused by why those issues are tied to serverless.
The only one that seems like it could even be related is the page loads being slower. But I'd be surprised if the issue were the serverless aspect of the architecture and not just that this software is apparently shoddy overall.
Are you by chance referring to A Cloud Guru? I have the same auth0 and loading experience when using their site, and I suspect it's built on Lambda, but I'm not sure.
It is indeed. They can't stop talking about it in their videos. That said, I used their resources, along with some other online resources to get my AWS Associate SA Certificate and I can say I was pretty pleased with the content.
I know of 1 large scale web application that is 100% driven by serverless technologies.
In the mid-90’s it was common to run httpd via inetd. That lasted until HTTP 1.1 came along. I see people running websites out of Lambda now and just get a sense of deja vu. We realised this start-on-demand style didn’t scale for highly interactive use cases 20+ years ago!
Count me in as a over-engineering and selling-things-engineers-do-not-need hater.
The very list of benefits on ZEIT, mentions 4 things: first two of them are clear over-engineering bloat (plus premature optimization), and second two were actually created only because of serverless. They were (and nothing more was mentioned in section benefits ;) :
* Clusters or federations of clusters
* Build nodes or build farms
* Container registries and authentication
* Container image storage, garbage collection and
distributed caching
I don't know why everybody can't see fakeness of argument'you need clusters, farms and hundreds of servers'. You don't. Actually you do only (contrary to your statement), if you're FANG.
Why? Because look at real world HUGE examples. E.G. stackoverflow (and no, your company/startup/whatever, will not reach their level of traffic) can do everything on literally dozen of servers, while they admitted that in some scenarios 1 web server was enough. Source: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...
Our 10x..100x smaller companies would perfectly do on 2..4. There is no need for whole over-engineering.
The ultra funny thing is ZEIT selling 'deployment self-heal' as old known (windows anybody) and ridiculed recipe: it will work after restart. Right. It's better to shut off car engine, go out, go in, and start again. This is XXI engineering :)
Facebook, Amazon, Netflix, Google. Jim Cramer created the acronym to name these four companies as high-performing tech stocks years ago [0]. But now, as in the above usage, it is more of a catch-all name for the big tech companies. And in its new role as a name for a category of companies, people confuse Amazon/Apple, and also implicitly include other equivalent companies.
Am I the only one having problems to follow .gif "demos"?
When I get to the image, it is in the middle of everything and I don't really have an idea what is going on. Even watching it multiple times, I am not sure where it starts, ends, what the individual steps are.
Or is it because I just don't know enough about this stuff?
asciinema is open source and does this really cool thing where it actually emulates a terminal. asciinema recordings are tiny compared to real video, and get rendered as text by your browser, so everything is sharp and snappy.
You can view controls on GIFs in many cases as well, I believe.
Also, nobody seems to make use of these features, but GIFs do not have to loop, nor do they have to loop by going back to the very beginning of the animation as well. In particular, I find it extremely ineffective where there are GIFs that only show a "final product/scene" for like 1 frame before looping back to the beginning.
I've not had the opportunity to use this project yet, but https://github.com/nbedos/termtosvg seems to be another nice alternative. It provides the option of using templates that have playback controls or a progress bar. Also supports copying from the "terminal" like asciinema does.
I believe technically it requires lower bitrates to maintain the same quality (as mp4 etc), as opposed to just having de facto better quality. The lower bitrate is what results in less bandwidth usage.
Well yeah, you can get lower bitrate with the same perceived quality, or better quality with the same bitrate, or a mixture of both. It all depends on what options you set when encoding.
Chromium browsers compiled without proprietary codecs require webm. It's a small subset of people (I am one), but I just don't play and/or leave sites with mp4 only.
It is kinda-sorta free and open source, though (at least the decoding part). Cisco pays for their codec, and as long as you use their implementation you are covered by their license.
Specifically, to be covered you have to download a binary release from https://github.com/cisco/openh264/releases during the installation to be covered by Cisco's MPEG LA patent license, but they provide builds for most major platforms.
Thanks a lot for your feedback. I think adding playback controls on hover is going to help a lot.
We picked video/gif as the delivery format to give people a real-life impression of performance. (I took the videos myself while using the platform on a regular production plan.)
And because it's an actual gif rather than re-badged mp4 (gifv) it's 1.6mb, it doesn't allow you control via browser controls and doesn't have a clear indicator of length.
If it's not overdone, an animated walk-through can give you an impression of how simple something is - here a short animation can be worth a thousand words.
For more complex topics I find animations less helpful.
I do alot of gif documentation and its unfortunate that its hard to follow.
I only do this because its convenient using shareX and supported at the same levels images (no embedding needed), just traditional `img` tags. Also, it works natively with markdown so its convenient
The only suggestion is to download "Gif scrubber" or "Play this gif" or "Videoplayback Controller" off google chrome extensions
they do explain it. gifs have there place but this may be too long for a looping gif, i believe a proper stream would have been better to display the commands and process.
Yep. Pretty soon. You write code. You create a docker file. You find a place to run the docker file with your code [cheapest!]. Run it through your tests. Monitor it. The end. No vpcs,salts, puppets, sshs,chefs, horses,anisbles, cats,ec2s,devops,noops, sysadmins, kubernetes or chaos monkeys required.
Until you discover that the thing you are building requires more than a single application running in a single container and you end up building an entire "Operating System" around your containers and the circle starts all over again.
Complexity is hardly ever in the solution, but mostly in the problem. Single solutions to complex problems often ignore/forget important parts of the problem and they come back to bite you, hard.
I don't know... My experience is the other extreme - tech teams that make everything super complicated to support everything that can possibly happen. As a consequence, the it environment requires six months of experience to even understand. It's really not very fun to work in those environments. Lots of unnecessary complexity.
On the other side of the coin, all these things would make sysadmins absolutely loathe you.
- You have 20-30 specialized applications which aren't package based stored in an application share mounted at /app with the convention /app/name/version or something but then this tool hardcodes itself to /opt/name.
- Using nonstandard ports doesn't make you more secure against someone who's specifically targeting you, but it absolutely stops automated attacks and logspam. What do you mean you can't change the port?!?
- Because there's no such thing as a secure internal network. All sites go through the hardened proxy.
- Invest? Nobody is going to pay for certs for internal services and who wants to set up a reverse proxy If you can't use an internal CA that's just poor form.
- That's a new one for me. I assume because they have a single shared database for all of their apps.
There is actually a reasonable arguement for the stored procedure one. It means you can have different permissions for tables as you do for stored procedures so your application doesn't have permission to directly query your passwords. The benefit there is if an attacker gains access to your application and/or the DB authentication credentials they cannot then export users passwords or other sensitive information.
Seems more efficient to me to use column-based permissions in this case, and create stored procedures for the queries you need to interact with the "sensitive" columns.
That of course does increase (dba) administration overhead, but it seems on it's face simpler and far more "programmer efficient" than storing every single query as a stored procedure.
It's a difficult thing to generalise but if you have a pretty large or complex application and DBA (or several) then it does sometimes pay to have your DBAs write the SQL because it's a different enough paradigm from normal backend development that not all backend developers are skilled at writing perfomant SQL. However few applications are that complex, not all businesses can afford a dedicated DBA team, and there are clearly a lot of very talented developers who can turn their hand to multiple different stacks. So it's a difficult thing to generalise.
“won’t invest in trusted SSL certificates for internal services.” <== this, I shrug my head every time. I’d do something about it If I didn’t have a billion other things to worry about like working services from teams.
I have the same experience and I'm on both sides in my day job. On the one end preventing developers from having to make the same (learning) mistakes that have already been made by ops so they can focus on developing and delivering, stuff like HA, security, backups, networking, statefulness, etc. Just deploying you apps on containers doesn't solve these problems.
But on the other end I'm constantly fighting the complexity of existing environment and preventing to much new complexity being added. It seems that every time a new tool/product is introduced which makes things simpeler, it just end up being a Hydra and total complexity is only increased.
I still have to find the holy grail if there is one, some middle ground? Or maybe it's just human nature to always make complexity where there is none since "It can't be that simple, can it?".
today new devs, picking up a 10-30yr old system: this is garbage, we have to recompile under the new 4.0 kernel just to be able to run it on new servers and nobody know how to read a makefile! lets rebuild with containers!
tomorrow new devs, picking up a 5yr old project: this is downloading a centOS build from 7yrs ago! which only run on docker from 5yrs ago! and nobody know how to build a new base image with the libraries the old one have! lets rebuild it in <whatever snake oil they have in 5yrs from now>
the problem is not serverless or not, its clueless people and corporations with code/process rot. and the only lesson from the cycle, is to never trust people who proclaim one solution is the holy grail for all problems.
Setting up and maintaining a kubernetes cluster on my own would probably kill me desire to touch a computer again.
Azure and Google provides free Kubernetes-cluster-management-as-service. But you are right, its still a hassle.
Been going through that process the past few days on Scaleway, as they don't have a hosted K8. It's... tough... but I'm learning a lot and have a much deeper understanding of K8s clusters from an operational perspective now. I have a much deeper understanding of the magic happening behind the scenes to keep everything talking to each other, and even if I wind up on a hosted solution in the end, it's been invaluable.
thats what I told myslef too when I invested hours after hours learning Knockout then AngularJS. Only for reactjs to dominate a year after. I skipped learning react and stuck with angular. Now I hear rumours theres a new sheriff in town named Vue.js.
Careful with filling your brain with domain-specific knowledge.
Deploying to a Kubernetes cluster is easy. You just have to learn a few key concepts to write your own .yaml definitions. Also, some open source frameworks come with the .yaml definitions already.
Well, this is probably to find common ground between FaaS people and DevOps, haha.
Being a FaaS proponent, I still think this could be a good idea, because of the kinda standardized workings of Docker. I mean, AWS also uses Docker for local testing.
It feels so amazingly depressing for so much sunk knowledge to just go away and be worthless. Feels like I could have learned so many more things that I could have had gotten joy out of today and still extract useful things out of, or build things on top of, long into the future if I'd just focus on the time-invariants of knowledge space.
Half the stuff you described there solve different problems to docker. Not to mention that docker doesn't solve all problems in infrastructure.
I like to think of docker like git. It solves some problems with distributing data but it doesn't solve the problem of developers writing code, writing tests, pipelines, etc. Nor even problems with 3rd party APIs etc. Obviously this isn't a perfect analogy but my point is docker isn't a magical silver bullet.
I've worked in places where their solution to everything was "docker" and it honestly caused more complexity problems than it solved. That's not to say I don't like docker; in fact I've been a big advocate for containerisation long before docker was a thing. However like with any tech, the key is using the right tool for the right job instead of attacking every screw with the same hammer.
I don’t think that’s likely or desirable. VPCs and the ability to make a service that’s not in the public internet are hugely valuable from a security standpoint. And Ansible, Terraform, etc are great for automation and managing configuration and architecture. It’s unwise to discard everything that came before you when hopping in the shiny new bandwagon. Even if it is the future, last generations tools can still teach you a lot.
You still need all that, you're just paying someone else to do it for you. As the app gets more complicated you will have to do more. It is impossible to remove complexity by adding abstraction. You've only hidden it.
Whenever a solution offers you simplicity, you're giving up flexibility. That's why I'd prefer to use a more advanced but standardized open source platform like Kubernetes.
You use dat or MaidSAFE to write client side apps. The back end is end to end encrypted, secure, automatically rebalanced, uncensorable, permissionless, and people install your app and use a cryptocurrency to pay for resources.
And of course you use public key cryptography to maintain your app and upgrades propagate without needing to select a domain or host for them.
Looks great for basic websites but it's missing the biggest and most difficult piece of cloud infrastructure. The DATABASE!
Today you'd have to open up your cloud DB provider to the world since Zeit can't provide a list of IPs to whitelist. This is a showstopper for me unfortunately.
From what I've seen, when people talk about serverless there's 2 camps.
Functions makes life easier camp. Serverless development and deployments have nicer properties which make them easier to reason about and eliminate entire classes of errors.
Functions make edge computing possible camp. Serverless functions can be deployed in datacenters around the globe close to your users, offloading compute from your core and improving latency.
Now let's talk about DB semantics. You and many other people in the first camp probably want your business logic on strongly-consistent SQL transactions. That's good, it's the right semantics for the job. But it's incompatible with the edge model where the functions are decoupled from the central datastore.
So I think that you're asking for something the community isn't mature enough to provide yet. The momentum is towards unification where we need stratification (with respect to coupling).
If you start with the intention of asynchronous, non-transactional eventual consistency as your assumptive design model, you'll realise:
1. Most applications don't need 100% correctness. Few people are writing nuclear reactor control systems, or bank account management software
2. This model frees you to up to compute at the edge more.
Is there a chance that you consider something valid that eventually wasn't? Yes. But in a single actor-per interaction system (e.g. an single actor mutating something, multiple people seeing effects, etc.), typical to the web, this is OK.
People know about CAP theorem but hang onto the C with dear life. Let it go. It makes everything easier. Honest.
You make some good points, but I disagree that giving up consistency makes everything easier. There are trade-offs. In an eventually consistent system, in my experience, application code becomes much more complex to implement, test and debug.
Most databases are quite unfit for the serverless world that's becoming a reality, where the needs shift towards global replication, flexible horizontal scalability (sharding) and vertical (provisioned QPS).
We like and use CosmosDB because it fits this criteria. We anticipate that Google Spanner, CockroachDB and similar databases will become the go-tos in combination with ZEIT Now.
So what do most of your current customers do for data storage? I mean, I doubt they all use CosmosDB? (simply because it's not particularly mainstream)
We don't have insight on what our customer's use mostly.
We use existing technologies. Anyone can use any cloud Database service. We've datacenters on San Fransisco and in Belgium. So, based on those users can choose where they need to deploy their DBs.
Usually we recommend to configure databases via env variables. Users can also use our [now secrets](https://zeit.co/docs/getting-started/secrets) service as well to avoid hard-coding secrets.
You are going find most people still are using Master-Slave databases at some central datacenter.
Spanned databases are great, but most of the time performance is not there (It's getting there, but it needs to be there for a year before people start to care)
Frankly I blame the SQL DBs for the rise of NoSQL. They didn't move fast enough for this kind of environment, and stuff like Cassandra fit that need pretty well.
Talking about running things behind the scenes, mainframes with pre-SQL NoSQL DBs (ADABAS, IBM IMS) and even with no DBMS in a modern sense at all (running on TPF, working directly with Direct Access Storage Device records) are still doing very well.
At Cloudflare, we're working on expanding Workers (https://www.cloudflare.com/products/cloudflare-workers/) to allow access to your existing DB servers & offer protection with Argo Tunnel (https://www.cloudflare.com/products/argo-tunnel/). We are also enabling Workers to write into Cloudflare’s globally distributed cache, reducing retrieval time for repeated query results. We hope this will be a differentiator with using Cloudflare & highly valuable for your use cases.
Workers don't run Docker containers intentionally. The goal with Workers is to run with a lower memory overhead (~3 MB) and lower startup time (~5 ms) than you can get with full container isolation. This allows your Worker to run affordably in 150+ locations around the world. In many ways it's the ultimate destination for serverless, running code in a multitenant process where all you manage is your code.
Amazon is building an HTTP interface to Serverless Aurora to solve this problem. You can secure it via IAM rather than network segmentation, much like DynamoDB.
As far as protocol is concerned, if you're using TLS, a client certificate, and a strong password, sure, opening your database servers to world accessible should be fine.
The problem is that it's possible, and very likely, there are exploits in the wild for your database server -- that are known but you failed to update for a day, or are 0 day exploits -- which are exploitable without having an authenticated account. Those issues can't be exploited if you firewall your database server to known IPs, but once you make it world accessible, all bets are off.
You may be interested to know that Heroku exposes all PostgreSQL databases publically, and unless you are an enterprise customer there is no way to turn that ‘feature’ off:
How would that exploit get exploited if you have mutual TLS authentication protecting your database connection? Or are you saying it's likely that particular piece of the database infrastructure is likely to have an exploit?
Just using SSL connections doesn't cover it, for two reasons:
1. You can't force MySQL to only accept connections over SSL. You can only enable SSL, and set specific accounts to only allow SSL logins. This means that any sort of "unauthenticated attack" on MySQL will work -- if you can exploit MySQL without a valid login, enabling SSL for users won't help you.
2. Amazon RDS supports using SSL connections, and will issue your MySQL server an SSL cert from their certificate authority, so your client can validate the server. It does not, however, support client SSL certificates, for the server to validate the client. Which means the only thing SSL connection is doing for you is encrypting the connection -- it's not in any way validating the client, and anyone can download the RDS region's CA certificate and then connect/exploit your MySQL connection normally.
Yeah, if the database doesn’t support mutual tls, it is clear that wouldn’t be sufficient protection. Having a proxy in between client and server that handles this (e.g. envoy) would be a good option.
The problem with just using something like Lambda on a VPC (so you can use a traditional firewall to protect your MySQL server) is that the cold start times are 10-15 seconds, to get your local network interface and connect to the server.
You're suggesting that from the time a user makes a request to the function, the function should load up, and then create a client VPN connection to the database server, and then create a Database connection? Pretty sure that'd end up pushing towards the same cold start time. Also, no managed database provider is going to offer a VPN connection to it, so you're now definitely maintaining your own database server, when part of the point of getting into serverless is to not worry about servers.
That's the service that mlab.com offers (which we use and are happy with). It does mean you have to be a little careful about your queries though, and cache if necessary.
No, you do not make your DB publicly available only because you are using TLS. In general you do not expose anything to the world unless it is necessary and your service is battle tested. There are probably shitloads of possibly RCE-able vulnerabilities in the public facing code of most DBS, because, the heck, they aren't built to serve the public web, but to do database magic.
Sure, but I reached out nearly a year ago to the maintainers with no reply. It seems anyone can now execute anything on your infrastructure (for free!), which isn't great.
That's the case already: anyone can sign up for a free Zeit account and deploy code (wrapped in a Docker container) that then executes on Zeit's infrastructure.
> “A very common category of failure of software applications is associated with failures that occur after programs get into states that the developers didn't anticipate, usually arising after many cycles.
In other words, programs can fail unexpectedly from accumulating state over a long lifespan of operation. Perhaps the most common example of this is a memory leak: the unanticipated growth of irreclaimable memory that ultimately concludes in a faulty application.
> Serverless means never having to "try turning it off and back on again"
> Serverless models completely remove this category of issues, ensuring that no request goes unserviced during the recycling, upgrading or scaling of an application, even when it encounters runtime errors.
> How Does Now Ensure This?
> Your deployment instances are constantly recycling and rotating. Because of the request-driven nature of scheduling execution, combined with limits such as maximum execution length, you avoid many common operational errors completely.
Somehow this sounds very expensive to me (like restarting Windows 2000 every hour just to avoid a BSoD, except that here it’s not that time consuming a process) and seems to leave aside caching, state management and other related requirements on the wayside for someone else to handle or recover from.
Or it’s likely that I’ve understood this wrong and that this can actually scale well for large, distributed apps of any kind. Sounds like magic if it’s that way.
> I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say "Yeah it works but you're leaking memory everywhere. Perhaps we should fix that." I’ll just restart Apache every 10 requests.
> Somehow this sounds very expensive to me (like restarting Windows 2000 every hour just to avoid a BSoD, except that here it’s not that time consuming a process)
The crucial difference is that the activation cost of booting up a full OS (like Windows 2000) is massive (typically involving a chain-reaction of CPU and IO-intensive init services), whereas that is not the case for our serverless infrastructure.
What you do point out is one of the most important engineering challenges we had to solve (and continue to be focused on).
> seems to leave aside caching, state management and other related requirements on the wayside for someone else to handle or recover from.
Configuration state is the bane of server management. Ideally you want to maintain state in a system specifically designed for it - i.e. a database, or version control software.
The primary benefit of serverless is that you can pay per-second for CPU usage, right when a request comes in, instead of leaving whole OSes running all the time. This other stuff is just a bonus.
Awesome!
While I was at AWS Summit in NY, I asked a round circle of AWS ECS/EKS users (Container orchestration products) about thoughts on a Docker container service that could execute like a FaaS product and there seemed to be none anyone knew of. I have a portion of a legacy application that's used infrequently and too costly to decompose but works fine Dockerized.
I'm confused about pricing. I come from using AWS Lambda, where you pay for the amount of memory allocated for your function, how many times it runs and how long each run is.
Looking at Now, it looks like you are billed by the 'plan' that you choose, and that decides how many deployment instances you are limited to. What does a deployment instance mean for something that is 'serverless'?
EDIT: Whoops, I see that there are 'on demand' prices for deployment instances too--now I just need to figure out how deployment instances map to serverless.
So I've been messing with Fn + Clojure + Graal Native Image and I'm seeing cold start times around 300-400ms and hot runs around 10-30ms. TLS adds something like 100-150ms on top of that. I was excited about seeing improved docker start times, but it seems like you guys are pretty much at the same place I am with it.
Here's my question, being relatively ignorant of Docker's internals: _is it possible_ to improve that docker create/docker start time from 300-400 ms (all in) to <100ms? 300-400ms is kind of a lot of latency for a cold boot still, and people still do things like keepalive pings to keep functions warm, so it would be pretty great to bring that down some more.
There are a lot of low- and medium-hanging fruit in this space.
I take the view that there is a "warmth hierarchy" (an idea I spitballed with other folks in Knative before Matt Moore took it and filled it out a lot).
You have a request or message. You need to process it. The hierarchy runs CPU > Memory > Disk > Cold.
CPU-warm: immediately schedulable by the OS. Order of nanoseconds.
Memory-warm: in-memory but can't be immediately scheduled. Our reference case is processes that have been cgroup frozen. Order of tens of milliseconds.
Disk-warm: the necessary layers are all present on the node which is assigned to launch the process. Order of hundreds of milliseconds.
Cold start: nothing is present on the node which is assigned to launch the process. It will need to fetch some or all layers from a registry. Order of seconds to minutes.
Some of these problems can't be solved by Kubernetes (it doesn't know how to freeze a process), some of them can't be solved by Docker (it doesn't know about other nodes that might have warmer disk caches). Then there's the business of deciding where to route traffic, how to prewarm caches and based on what predictive approach, avoiding siloisation of processes due to feedback loops.
As these are knocked down they will pay dividends for everyone.
Hey there, fn dev here. Cool to hear about folks testing the graal images! We have done some experimental work and made some discoveries in the container cold start time dept. which we've documented https://blogs.oracle.com/developers/building-a-container-run... (see: Fast Container Startup).
The main things are optimizations around pre-allocating network namespaces, we've built some experimental code to skirt around this https://github.com/fnproject/fn/blob/master/api/agent/driver... (which can be configured & tested) and we're testing approaches like this to attempt to get start times down to as low as we can. <100ms still seems like a lofty goal at this point, but this is becoming a pain point for various FaaS platforms that are container based and some optimization work should come out of all this. A few layers up, it's a little easier to speed things when throughput increases for a given function, but devs do seem to want the 0->1 case to be almost as fast as the 'hot' case so we FaaS people need to answer this better in general, it isn't something we want users to have to think about!
I am already running an API service on Zeit now using a golang container that adds the binary and a csv file to a scratch image and re-reads the csv on each request (all requests take less than 0.3 seconds so i have not optimized).
Currently I have to make sure to set min instances to 1 on the 'production' version of the API and set min instances = 0 to older versions.
Will have to try before knowing for sure but I seems like switching to serverless docker would mean I don't have to make the distinction between 'old production' and 'new production' anymore and keep servicing requests to very old versions of my API without an expensive (7 seconds) cold boot.
I did not play with it, but this actually looks pretty cool. The documentation seems to be sane/good quality.
A few questions: How would one coordinate between multiple nodes running an app? (scenario in which the nodes cooperate to find each other. is there some sort of discovery available?) For the docker case, does it support health probes? (how you you know if the app is healthy?)
For the duration of the warm phase (i.e.: while the process is running), we ascertain healthfulness via `SYN` packets as a baseline.
Because you give us your code under the constraint that it will expose data over a port, we have a much better safety baseline than stock schedulers, that operate under the assumption that the process describe by `CMD` is a black box.
We will describe this in more detail in coming weeks. One of our priorities is to minimize the cognitive load on the developer necessary to run a healthy service at scale.
This is truly amazing and has so much baked in I find myself wanting to use it. However most apps my team and I work on require some form of persistence - uploads, log ins, API rate limits, etc. These scenarios don't fit well with the serverless world. It is technically possible to offset most of this to S3, a hosted NoSQL and perhaps even a service for dealing with a images/thumbnails. By that time your monthly bill is in the three digits, though. For a big web shop this is fine but your average freelancer with multiple smaller and finite-resource projects it's something to consider carefully.
> A new slot configuration property which defines the resource allocation in terms of CPU and Memory, defaulting to c.125-m512 (.125 of a vCPU and 512MB of memory)
Sure sounds like needing to be concerned about the hardware. That feels like a leaky abstraction that the serverless design pattern claims or appears to take care of, but seems like it doesn't in practice. Is "serverless" the right level of abstraction? I'm not sure.
Whether it's serverless of not choice of hardware is very important.
For an simple web app, you might not need special CPU and Memory requirements.
But for a video encoding/decoding serverless app may needs different CPU/Memory requirements (and GPU).
That's what this `slot` configuration property does.
The default is good enough for all the general purpose apps. But if you need more, you can control it.
I agree that one-size-fits-all doesn't work with hardware requirements, but I still think this is a leaky abstraction. The promise of Serverless is not having to care about servers, but this is an option to specifically control that. A better abstraction would auto-optimise hardware spec for the software's performance characteristics - that would be much more "serverless".
This development was inevitable, especially since most Functions-as-a-service infrastructure was really docker (or some other) containers running in the background, spinning up on demand using a pool of servers. Azure Container Instances launched last year, and now Google Cloud Functions has serverless containers in alpha, along with the serverless addon for GKE/Kubernetes, and there should be a good bit of announcements this year as other providers follow.
The max time limits still seem pointless to me, there should be an option for unlimited uptime so the full spectrum of ephemeral function to entire app can be managed and deployed using a single flow.
There is no 'hype' word in history I hate more then serverless. I was fine with microservices and all those other hype words but serverless is terrible.
The interface it gives you is the functionality that the server provides. A server does not mean 'operating system'. you don't have to deal with the OS, but you still have you app running on the server.
> 1. Once your docker container is built, am I paying to store it on Zeit?
No. You don't need to pay to store containers.
> 2. How is a Dockerfile versioned? Can I update it? Or do I need to redeploy
Once the Dockerfile is built, you'll get a URL for that. It's a immutable URL and you don't need to re-deploy again.
It'll sleep if there are no HTTP requests. If there are it'll start again.
> 3. Is pricing for containers granular by time or per request to a container?
It's based on the container running time. But we'll have a another blog post on this soon with more info.
> 4. How can these Dockerfiles talk to each other? Is there an API or method to fetch specific url's for each container?
There's no internal API to communicate with each other. But every docker container has a URL. You can use our [alias](https://zeit.co/docs/features/aliases) feature and communicate with each other via HTTP.
This is more of a question to @rauchg and others who have used now one. When I use now within the AWS realm, are the AWS services when used in conjunction with Zeit Now containers subjected to ingress/egress costs? As it does sound like I am using a different cloud as far as AWS is concerned
Is there support planned for using this in combination with docker-compose? If so would be sweet to see this integrated with Prisma - https://www.prisma.io/
How can something be serverless if it habdles http requests? Does it just process requests and never serves responses, or is it just a hip-sounding misnomer?
Long story short, there's a server somewhere, but most of the configuration is abstracted away. Instead of thinking of servers, they want you to think about endpoints (which they manage somewhat efficiently for you).
We will be announcing very clear pricing when it goes into GA. Needless to say, serverless makes for a much much cheaper (and robust) operation than VPS and traditional clusters.
If you pull an application apart and squeeze it into managed services, your costs can drop by an order of magnitude depending on your availability requirements, engineering related costs, etc. A good article written by Spotinst's CEO [1]
Not trying to shill, not everything should be serverless, and I should work to be more platform agnostic, but AWS has been great to and for me, AWS marketing material [2]
Care to elaborate? Did you write your own container builder (for example like Google did with kaniko) that you use instead of docker? If not, I fail to see how you can build faster than me locally.
Now uses Docker so you just need to also use Docker to run your container, there is nothing special to do to run it in locally or offline since we don't require you an specific runtime or API. Just a normal HTTP server running inside a normal Docker container.
I've never been more excited about a cloud platform than I am about Zeit/Now. I wish you offered stateful volumes or key value stores or something where I can persist my data so I can keep all of my stack on Zeit.
I think “serverless” doesn't actually mean no server (a distributed system, like Dat); it means “don't worry about the server; we'll take care of it for you”.
…or maybe that's “Cloud”.
Is there a clear explanation of what “serverless” means, and how it's different from “Cloud”, suitable for semi-technical non-developers?
From what I understand, "Cloud" basically means that you still have to deal with servers, whether they be VMs or physical devices, but all of the networking and loadbalancing aspects are abstracted away. "Serverless" takes it a step further and makes it so you don't even have to deal with setting up VMs, you just give the provider the code you'd normally run on a host, and they'll automatically handle all of the infrastructure stuff, like spinning up extra hosts under heavy load.
It's usually called "serverless" when the "servers" are created on demand when a request comes in, cold server start-up takes less than a second, and there's theoretically no server state between requests.
- "sub-second cold boot (full round trip) for most workloads"
- HTTP/2.0 and websocket support
- Tune CPU and memory usage, which means even smoother scaling
And all that for any service you can fit in a Docker container - which is also how you get near-perfect dev/prod parity, an often overlooked issue with other serverless deployment techniques.
On top of all that, ZEIT has one of the best developer experiences out there today. Highly recommend trying it out.
And for the perpetual serverless haters out there: this is not a product for you, FANG developer. I think people underestimate how well serverless fits for the majority of web applications in the world today. Most small-medium businesses would be better off exploring serverless than standing up their own k8s cluster to run effectively one or two websites.