This is probably the best annoucement of a new feature I have ever read. It makes an analogy to an existing technology. It provides a clear description of the new feature. It provides clear examples of how to use the new feature with a link to a sandbox so you can run and modify the examples. And it explains the thought process behind the implementation. In additon, I didn't notice a single typo, spelling or grammar error.
This is really interesting. I have an idea where this could be helpful to the Plex user community. Recently Plex added a header that blocks the page from being iframed (X-Frame-Options).
Would doing something like this, obviously replacing example.com with their own domain.com, replace the offending header?
addEventListener('fetch', event => {
let request = event.request;
if (request.headers.has('X-Frame-Options')) {
let newHeaders = new Headers(request.headers);
newHeaders.set('X-Frame-Options', 'ALLOW-FROM https://example.com/');
event.respondWith(fetch(request, {headers: newHeaders}));
}
// Use default behavior.
return;
});
Of course, you could only apply it to your own server.
Also, you would want to think carefully about clickjacking attacks (where someone puts your site in an invisible iframe and tricks people into clicking on it). The X-Frame-Options header was probably added to prevent clickjacking.
Of course this would be the user's personally hosted server. Typically hidden behind a password and loaded in some sort of HTPC manager like Organizr.
According to reports, Plex' intention was to prevent clickjacking, which is perfectly reasonable but left many of their users from being able to use their Plex servers within the HTPC managers.
Congratulations on the announce - it seems like a game changer of a feature, and one I look forward to testing.
Will Cloudflare be curating a list of useful worker scripts? I imagine there will be certain usecases that get a lot of attention (e.g. ESI)
Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
And finally, does this change Cloudflare's role as a "dumb" intermediary?
> Will Cloudflare be curating a list of useful worker scripts?
Better than that, we plan to unify this with Cloudflare Apps, so people can publish a reusable script as an app, and other people can then "install" that app onto their site with a click.
> Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
The worker sits in front of the cache, so subrequests made by calling fetch() will go through the usual caching logic.
Eventually we also plan to expose the "Cache" API from standard Service Workers to allow you to manipulate the cache directly for advanced use cases, but this shouldn't be necessary for most people.
> And finally, does this change Cloudflare's role as a "dumb" intermediary?
That sounds like a policy question, which isn't my department. ;)
I looked a bit at the code in Chrome but determined that it was too attached to Chrome infrastructure that didn't make sense in our use case. Also, there are a lot of parts of Service Workers that don't make any sense outside the browser, so it looked like it would be pretty hairy trying to pull those apart. So, we're building our own implementation (a lot of which is still in-progress, of course).
I actually built a little V8 API binding shim using template and macro metaprogramming that I really like, which allows us to write classes in natural C++ and trivially export them to Javascript. We've been filling in the various non-V8-builtin APIs using this.
We're using libkj's event loop, HTTP library, and other utility code. (KJ is the C++ framework library that is in the process of spinning out of Cap'n Proto. Yeah, I may be suffering NIH, but I think it's worked well.)
A bindings layer for v8 that was specifically intended to make implementing web-style APIs outside of Blink easier. At the time at least, refactoring things like SW out of Blink was ~impossible.
Darn, I wish you'd seen it too :). Sounds like you came to similar conclusion I did -- using C++ templates to stamp out v8::FunctionTemplate and ObjectTemplate instances.
I was pretty happy about how Gin turned out and always thought it would be nice to put it on Github where it would be easier for people to reuse.
But at the time Chromium still didn't allow all the modern C++ features, so Gin had to depend on base::Callback, which made it hard to extract cleanly.
Pretty awesome stuff, will echo that this is one of the cleanest feature pages I've seen.
I'd love to hear more about your evaluation of Lua. LuaJIT is so blazingly fast(and small!) that I'm sure it'd be some pretty significant compute savings.
What sandbox solutions did you look into? Separate lua states, just overriding ENV/setfenv() or something completely different?
We love Lua and LuaJIT -- we use them extensively here.
But for running third-party code, we need to everything in our power to reduce the risk of a compromise.
Every sandbox (including V8) has bugs, and security is about risk management. With scrutiny, the low-hanging fruit is found and the risk of further bugs steadily decreases. At the end of the day, no Lua sandboxing mechanism has had anywhere near the scrutiny of V8. It's a totally unfair chicken-and-egg problem: to get scrutiny you need usage, but to get usage you need scrutiny. But, it is what it is. :/
I think there's definitely a compelling reason to use JS both from a developer comfort perspective and the fact that it's a pretty battle-tested path.
It sounds like anything short of a full-blown container(with all the overhead that brings) wouldn't be sufficient to cover the security concerns. I'd love to see Lua battle-harded a bit more in that area so was just curious if you had any new learnings.
Are you able to talk about the pricing structure? Is this going to be per request, total cpu time, etc?
edit. In another reply you said that pricing hasn't been finalised which is understandable. We've got a few use cases which CF Workers would be ideal for but we'd be looking at 10-15k requests per minutes which could get expensive if pricing is per request.
We are taking input on what the use cases are in order to come up with a pricing model that makes sense. Definitely reach out with what you're trying to do and we'll take it into account as we're designing how it's priced.
Yes, we intentionally disabled it for now because the WebAssembly API allows dynamic code loading at runtime. We want all code deployed on Cloudflare to go through our standard deployment process so that we have copies of all of it e.g. for forensic purposes. Also, when we integrate with Cloudflare Apps, that code needs to be moderated.
What we'll do at some point is allow you to provide a WASM blob along-side your script, which will be loaded separately and then exposed to your script probably as a global variable.
In the blog post you talk about the trade-offs of using JavaScript vs other options like containers. I thought you might be interested in this comparison of using JavaScript vs other sandboxing options.
> Hosts of V8 can multiplex applications by switching between contexts, just as conventional protection is implemented in the OS as process context switches… A V8 context switch is just 8.7% of the cost of a conventional process context switch… V8 will allow more tenants, and it will allow more of them to be active at a time at a lower cost.
Yes. Currently you get at most 50ms CPU time per request, and memory usage of your Javascript VM is limited to 128MB. These numbers may change before launch.
In practice, though, a typical script probably uses much less than 1ms of CPU time per request and probably needs only a couple megs of RAM. Because we're applying limits on the level of a V8 Isolate, not on an OS process, the limits go a lot further.
Keep in mind that Cloudflare Workers are not intended to be a place where you do "bulk compute", but rather where you run little bits of code that benefit from being close to users.
I would very much like to see a pay-as-you-go cloud pricing model that allows you to pay to go over these limits (within reason.) For the use case I'm envisaging, filtering largish buffers of Cap'N Proto data (I'm tickled that both of these things are your babies!) I can use streaming fetch API to stay within the memory limit, but the 50ms limit would really be a hard wall that would cause requests to fail on larger amounts of data. I'd be happy to pay in those cases for extra execution time.
Currently, an error, because default handling might not be correct in many cases -- imagine a script implementing access control or expanding an HTML template. But I'd be open to the idea of letting the script call, like, `event.fallbackOk()` upfront to specify that default behavior as a fallback is safe.
Note that the 50ms is CPU time, not wall time. This is actually a pretty huge amount of time for handling one request.
Oh yeah, BTW: If you're an experienced systems engineer interested in working on a young codebase written in ultra-modern C++, let me know (kenton at cloudflare).
V8's API is C++. While bindings exist in other languages, they don't always expose everything and don't always keep up with V8's frequent API changes. I felt it was essential that we bind closely to V8, being able to use all of its features, and that we be able to track closely to the latest stable release.
Also, I personally have decades of experience in C++ and wasn't sure this should be the project where I work on learning Rust.
That said, I love Rust and highly encourage more people to use it.
Just for reference for people reading this: rust-bindgen can produce bindings to C++ code these days, though that gets you an unsafe library; you’d want to build another thin layer on top of it to nail down the ownership semantics so that you can provide a safe interface.
bindgen’s C++ support is better than it used to be, but still pretty limited, e.g. no support for inline functions, or their moral equivalent, methods defined within a class definition (which most C++ code has a ton of). And even if it works, to quote the documentation:
> using [C++] types in Rust will be nowhere near as nice as using them in C++. You will have to manually call constructors, destructors, overloaded operators, etc yourself.
Thanks. Sometimes I feel like the "just use X" folks don't really think about the problem at hand, not to mention how trivializing it can be. And this is coming from a Rust fan and someone who uses bindgen.
Policy is not my department, but technically speaking... I think a VPN service would probably be more effective running on AWS or Digital Ocean or whatnot.
So, there is the HTTP cache. It turns out a lot of storage use cases are actually answered by the cache. E.g. say you need some lookup table that changes every five minutes -- make it cacheable, max-age=300, and have your worker load it from cache.
Of course, that's not the answer to everything. We don't plan to offer other storage in v1 but we are definitely thinking about it for the future.
Kudos for re-using an existing API when one was already available in the same language for a very similar usecase.
Any time you commit to someone else's API -- whether it's an actual industry standard, or simply some de facto widely used paradigm -- you incur risks; conversely, now that you're a vested participant, consider being involved in the future of the spec so it can evolve where it needs to meeting emerging needs around its new uses.
Absolutely! One reason we wanted to get this out in public before it's ready is so we can properly engage with the spec writers (and the V8 team).
That said, I am amazed by how well the spec fits as-is. I don't usually like other people's API designs but in this case I think they did a really good job, and I've been pleased not to have to think about API design myself.
This is fantastic. Fastly is great but hard to use with Varnish VCL, and no other CDN had any real scripting capabilities. The service worker API is also a lot better than serverless cloud functions or lambda@edge.
Looking briefly, it looks like it's based on creating separate contexts, but not separate isolates. Contexts within the same isolate can be reasonably secure (it's how Chrome sandboxes iframes from their parent frames, after all), but they still share a single heap and must run on the same thread. Isolates can run on separate threads. We prefer to give each script its own isolate, so that one script using a lot of CPU does not block other scripts. We also want to be able to kill a script that does, say, "while(true) {}".
So yeah, it looks like a neat library but it probably wouldn't suit our needs.
This sounds like a really powerful feature. I love how it uses JavaScript, which makes it much more approachable for web developers.
My experience with Service Worker APIs hasn't been very positive, although I don't have any suggestions for ways it could be improved, so I apologize for the non-constructive feedback. Maybe after using it more I'll change my mind. I recognize that everyone involved is likely working hard to provide an API that's capable of handling a wide range of problems, many of which I likely haven't even considered.
Here's a more actionable complaint: fetch doesn't support timeout or abortion. I have a hard time understanding how this isn't a problem for more people. Say what you will about XMLHttpRequest, at least it supports these basic features. As an end-user, I always find it absolutely infuriating when things hang forever because a developer forgot to handle failure cases.
I'd love it if you published a locally runnable version. Aside from making it easier to configure and experiment, it would give me peace of mind to know that I could continue to use the same configuration if Cloudflare decided to terminate my service.
Could this be used for HTTP Push across domains? I would love an easy way to take advantage of HTTP Push for assets hosted on S3 without routing requests to my origin server.
This is very, very awesome work. My team has been working on enabling the PRPL pattern[1] and differential serving on a few platforms, and edge caching has been a problem. We've tried to use Vary: User-Agent, but that leads to low cache hits. This API would let us to much smarter UA sniffing at the edge.
From there we just need to parse some responses like JS and HTML, to be able to push their sub-resources, for an instant speedup and great caching for fine-grained deployments.
Http/2 push is interesting here. Beyon static assists I wonder about read API orchestration or equivalent of graghQL on edge. I.e a single request comes in and it's broken apart against catchable resources and pushed back to the client from cache where possible.
That's actually one of the canonical use cases we thought about while designing. It seems splitting graphql queries into cacheable chunks is a common need.
It's actually somewhat different. AWS Lambda is intended to act as your origin server. Generally your Lambda functions run in a small number of locations, not necessarily close to users.
Cloudflare Workers will run in all of Cloudflare's 117 (and growing) locations. The idea is that you'd put code in a worker if you need the code to run close to the user. You might want that to improve page load speed, or to reduce bandwidth costs (don't have to pay for the long haul), or because you want to augment Cloudflare's feature set, or a number of other reasons. But, generally, you would not host your whole service on this. (Well, you could, but it's not the intent.)
We haven't nailed down pricing yet, but we've worked hard to create the most efficient possible design so that we can make pricing attractive.
Can I use this together with CloudFlare streaming to open up stream ingestion based on some HMAC signed parameters?
To give regular end-user the rights to e.g. 'record one video of up to X seconds and Y mbps under ID "some-uuid"'
I've been waiting for a proper Content Ingestion Network for ages by now... if the CloudFlare video team ever wants to talk to someone who hand-rolled their own single-node version of this I'd be more than willing to share my experiences.
If you added this with RTMP+WebRTC ingestion support all of these [1] companies would probably jump on the opportunity... and dozens more would start up in that space ;-)
That is correct. However, the design is very different. Lambda@Edge is, from what I can tell, based on running a full Node.js process in a process-level sandbox for each customer, whereas we are embedding V8 directly.
What are you embedding v8 into exactly? What happens if I figure out a buffer overflow in v8? What do I break out into? Is it solely process isolation? Is it in a container? A VM?
> What happens if I figure out a buffer overflow in v8?
You report it to Google for $15,000. ;)
More seriously, though, this is something we've spent a huge amount of effort worrying about. There are many layers of sandboxing and mitigation, including an incredibly tight seccomp filter, namespaces, cgroups, ASLR, architectural changes to keep sensitive secrets away from risky processes, etc. But we still worry and will continue to add more safeguards.
Same as any other site setting, all of which need to be pushed to every location. We have existing infrastructure for this that propagates changes everywhere within a couple seconds.
We're certainly aware of them, but haven't spent a lot of time focused on this issue yet. Of course, the issue exists on all forms of shared compute. So if you're going to do crypto, you'd better make it constant-time. Which is... not easy in Javascript. (But we will provide the WebCrypto API, which might help.)
There is a theoretical solution that we might be able to explore at some point: If compute is deterministic -- that is, always guaranteed to produce the same result given the same input -- then it can't possibly pick up side channels. It's possible to imagine a Javascript engine that is deterministic. The fact that Javascript is single-threaded helps here. In concrete terms, this would mean hooking Date.now() so that it stays constant during continuous execution, only progressing between events.
That said, this is just a theory and there would be lots of details to work out.
> There is a theoretical solution that we might be able to explore at some point: If compute is deterministic -- that is, always guaranteed to produce the same result given the same input -- then it can't possibly pick up side channels.
Doesn't this require the timing and interleaving with other processes also be deterministic? ...which seems hard to guarantee with modern CPUs, async IO, and shared execution.
If you don't provide (real) time as an input to the program, then non-determinism of time does not affect determinism of the program. At least, in theory. It's definitely a long way from there to practice.
That's true -- but "the program" is also any code using yours as a middle-step, which is the case of a network service is a huge number of people. So anything that requires total program control rather than subroutine enforced safety is a nonstarter.
The timing may occur outside of your control, and then statistical profiling used. If the timing of a reponse to a network request has a time component that depends on shared load, then you have a side-channel.
I read the article, but I'm not sure what this technology is enabling really. Can it be thought of as a new player in the lambda/serverless category? Does it have any advantages to using other serverless stacks like aws lambda or azure?
The advantage is that the code runs close to your users. 90% of the people on Earth are within 10ms of a Cloudflare location, so by pushing your code out to the edge you can make your site faster. You also save on the cost of transmitting data around the world.
Sounds like this could be a pretty trivial way to load balance React prerendering. As long as the react code fetches all data in 1 call it should be at least as efficient as doing it all in Nodejs on your server.
- Is it "Cloudflare Workers" or "Cloudflare Service Workers"?
A "Cloudflare Worker" is JavaScript you write that runs on Cloudflare's edge.
A "Cloudflare Service Worker" is specifically a worker which handles
HTTP traffic and is written against the Service Worker API.
Consufing naming convention. Now you have to say 'worker worker' or 'non-service worker' so nobody has to wonder if you meant 'service worker' when you only said 'worker'.
Not really, because once there are workers other than Service Workers, they'll have their own names. To be clear, a Service Worker is one kind of Worker. At the moment it's the only kind, but we could introduce others in the future. For instance, maybe we'd introduce a "DNS Worker" that responds to DNS requests.
Also note we didn't invent these terms. "Workers" and "Service Workers" are W3C standards.
Oh my. Is there any form of persistence, present or planned?
If there is - this means that there (eventually) will be a way to have logs from the edge servers. I'm just thinking about a worker that would collect the requests and responses data in some circular buffer, and try to push it to the origin server. Eventually, the data will get through, so no CDN-returned 52x ("web server is not responding" etc) errors would go unnoticed.
We won't offer writable storage at the edge in v1, but you could always make a background request to push these events back to some log service you maintain (on separate infrastructure from your main service to make simultaneous outages unlikely). Note that you can make background requests that happen after the main response has already returned (but still subject to the per-request CPU limits, etc.).
We're thinking about how to do writable storage, but it's tricky, since one thing we don't want is for developers to have to think about the fact that their code is running in hundreds of locations.
Render your SPA (different index.html) when a login cookie is set and otherwise render your landing page (yet another index.html)? - Such that that my http://example.com can always be cached (unless it needs to hit the server where the same logic is implemented).
And in general, how do you manage your landing page vs. your SPA?
Some way to implement cache tagging and fast invalidation based on tags (a la Fastly) would be good. Without storage I can't see a way to keep a hash of currently valid tags - but I will play!
Great addition to your services.
I really want to see the pricing.
Also I'm quite excited to see how this will play along with the apps concept. Do you plan also to introduce a monetisation system for the apps? It will be a good incentive then for developers to get some nice income.
As a matter of fact, the monetization part already exists: Cloudflare Apps is a thing today, including paid apps. Today, though, they are limited to injecting client-side code. Once Cloudflare Workers are integrated with Apps, they'll be able to include server-side code too. https://www.cloudflare.com/apps/
Yes, it's my hope that Service Workers will be recognized broadly as the standard API for this.
FWIW I've seen several different folks saying they're working on implementing a Service Workers API shim on top of Node, which would be a great way to get an on-prem version, and should not be very hard to do.
Sounds like it. I think Cloudflare's edge (no pun intended) is going to be their 110+ locations to host this. As opposed to Amazon's 93 I believe? (If all cloudfront spots support lambda@edge that is) https://aws.amazon.com/cloudfront/details/
I would say at 117 edge locations vs 93 Cloudflare will have lower latency, but most notable in markets that most companies care least about. Probably of more importance is by using embedded V8 as opposed to a process-level sandbox running Node.JS, the Cloudflare service worker will have a much faster startup time in all locations. I seem to recall from tests in Lambda that we're looking at 40-50ms, which is actually substantial given how close these locations often are to the user. But the biggest edge in my books is CloudFront bandwidth is ridiculously expensive and Cloudflare doesn't charge (directly) for bandwidth. Bandwidth often being the single biggest line item in your AWS bill if you get a lot of traffic.
Technically, yes, but as I understand it, they frown upon it and could potentially terminate your account. See this section of their ToS (https://www.cloudflare.com/terms/):
>SECTION 10: LIMITATION ON NON-HTML CACHING
>You acknowledge that Cloudflare’s Service is offered as a platform to cache and serve web pages and websites and is not offered for other purposes, such as remote storage. Accordingly, you understand and agree to use the Service solely for the purpose of hosting and serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Cloudflare’s Service is also a shared web caching service, which means a number of customers’ websites are cached from the same server. To ensure that Cloudflare’s Service is reliable and available for the greatest number of users, a customer’s usage cannot adversely affect the performance of other customers’ sites. Additionally, the purpose of Cloudflare’s Service is to proxy web content, not store data. Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited. You further agree that if, at Cloudflare’s sole discretion, you are deemed to have violated this section, or if Cloudflare, in its sole discretion, deems it necessary due to excessive burden or potential adverse impact on Cloudflare’s systems, potential adverse impact on other users, server processing power, server memory, abuse controls, or other reasons, Cloudflare may suspend or terminate your account without notice to or liability to you.
I think in practice, unless you have a very popular website or are abusing it in some way, they probably wouldn't care or even notice. But you'd still be taking a gamble.
(I don't work at Cloudflare though so take whatever I say with a grain of salt.)
Wow, thanks for that info! Our enterprise account uses CF as a CDN for medium sized (~50MB) video files and I just assumed that they were ok with everyone doing that. I guess not.
I guess the answer for the OP then is, get an enterprise account and you can do whatever you want.
A more granular way to block requests than what cloudflare provides (IP, BGP AS, country)
Tokenizing credit cards at the edge if you have a payment provider that supports that, and are using CF's PCI compliant environment.
Injecting a unique ID into requests for log correlation. You get collisions if you do this via browser javascript for various reasons. I'm assuming v8 has a better Math.random(), or that you would at least be able to find a workaround since it's in one stack.
I worked for an analytics company at one point and we generated impression IDs server side (you'd be surprised how broken client-side JS randomness can be) and included them in our loader. If we had been using CloudFlare I would have put our loader directly into this service worker so that we could shave a few more milliseconds off our intial response time, so that we could get to our cached JS faster.
Also, this feature is pretty cool!