If fly.io is serious about their marketing, this is not the way to do it. Find a really slow website that sells something, give them your service for free and run half the traffic through your fast version vs their slow version. Show me how much money they made in increased sales vs the amount of time it took to implement and the ongoing costs of your service. I'm pretty confident the math won't work out, but I'd be open to being convinced. Trotting out the 10 year old Google and Amazon studies about sub-1% gains isn't the way to do it.
For the vast majority of startups or websites, there should be 50 things on your to-do list that have the potential to increase your conversions more than 1-2%, and your time is almost certainly better spent working on those things than adopting a service like this. I know engineers like taking the same thing and making it faster, it is fun and satisfying, but it almost certainly isn't the best use of your time when it comes to the bottom line. And that is assuming making your site faster actually will increase conversions by any meaningful amount, which is a big if (very old and very specific studies be damned!).
This is really just meant to be decent content, not a sales pitch. We obviously want to grow our business so we write about stuff that's close to what we do. Most people are really uninformed about site performance (still), you'd be shocked how many people don't even know about the 10 year old studies.
Also note, Amazon's -1% per 100ms of latency holds up for the ecommerce companies I've worked with. It's not a small number for any company, and we're actually really cheap for companies with a high value per user (like ecommerce! or saas!). You probably wouldn't serve billions of page views through fly with very cheap ads, though.
That said, I pretty much agree with you. We'll only succeed if we can keep proving that we're valuable. It's part of why we give enough traffic away for free that you can legitimately test us out. I don't particularly want people using us if they don't get any value out of it.
About 5 years ago, I undertook a significant web performance effort on an IR top 100 e-commerce property. We sped the site up significantly and, being the good data nerds that we were, split-ran the hell out of it and found a zero-point-zero (sub-noise) difference in average session value, conversion rate, and average order value.
Believing those then 5-year old studies, we went in the opposite direction and intentionally slowed the site down in multi-variate testing. Out to 1000ms of intentional slowdown per dynamic page, we saw zero-point-zero change in ASV, CR%, and AOV.
We kept the faster experience, because "hey, why not?" and we'd paid all the NRE for it anyway, but the project was a technical success but a big bust versus business case.
That's really fascinating! What about things like Google ranking that's supposedly also affected by page speed? (Or maybe it's not? I have a very limited SEO knowledge)
Agreed (naturally), but also be aware that the relationship may not be linear. There is quite possibly a "knee" in the curve where faster than some threshold will all be the same.
However, it's a quite easy test to run, so may be worth doing even if you can't stake your life on the results applying to a hypothetical speedup.
You don't mention the name of the company, but may I ask whether they were selling goods in the same price range?
I am asking because I can imagine that people who are looking for a $5 item are more inclined to abort the search if it feels to slow while I assume someone looking to buy a $1000 item will persist.
It was vistaprint, so item pricing is similar ($5-50 typically). The confounding issue is that because everything we sell is customized by our customers, they may be less sensitive to small differences in page speed. (If it takes 15.5 minutes instead of 15 minutes to design a business card end-to-end, maybe that matters less than a web or product search.) This was also 5-years ago, so our mobile experience was very much limited to order/account inquiry and everything else we steered you heavily back to desktop.
Our analysis (for this project) was also limited to session and repeat session analysis, so we would have excluded any effects related to long-run increases in sessions from distinct shoppers (organic search traffic increase or cross-shopper recommendations). I believe those were small enough to properly ignore.
I'm definitely not arguing that page speed is bad, but do question the gospel that every 100ms is worth X% more traffic and conversions.
That was the problem I had with this article as well. I've seen some of this stuff before, but I kept expecting to run into a graph showing various delays versus revenue on an e-commerce site. I just assumed it had been measured.
Instead when it came to revenue it was all implied (more people will stay on your site, they'll browse more pages, etc.) or an old statistic from someone else.
This second paragraph resonates with me and how I run my business. I was prospected for years that I should use a “said” managed DNS provider because my business would be doomed if DNS lookups for my web properties were under a certain ms timeframe.
Never mind trying to convince improper behaving resolvers out there to honor a TTL.
There are so many other things we need to focus on to convert users to paying subscribers. A 50ms DNS lookup, outside of what could be delivered by a zillion caches, is not on my list. Sorry,
Interesting; I always assumed faster performance was going to mean (lots) more conversions. Makes me wonder if AMP on google serps actually increases bounce rate due to lack of investment in a page load. There’s probably loads of stuff going on here and this area still needs more research. YC must have internal info on the tests companies have done to increase conversion/engagement? If not it’d be good to try to collate different experiences and do some meta-analysis.
I can say that 1-2% speed improvement helps conversions. We can see it in the graphs when the API slows down, people don't sign up.
Perhaps the context of the service plays a role - for us were are convincing people who might be a little more reluctant to sign up while other services are in high demand and no amount of friction is going to prevent a sale or signup.
Not true. Faster isn't _always_ better. If some features seem to take a lot of time, it appears to the user that it's "working". Take a website I hate a lot - TurboTax. A lot of the wait times between transitions is incredibly long. But I'd bet money that it increased conversions.
Yes, it's more of a UX issue than an engineering one. But users don't give a shit. If something appears slow, no one knows/cares if it's an animation or a slow server.
Sigh... Thinking up a different situation (or even an exception) doesn't make a general statement "not true". We aren't arguing mathematical theses here.
It is true that Intuit inserted some explicit delays to give the impression that their software is "working" really hard to deliver value. But, TurboTax is still very snappy in response to user action. That response is often an intentionally long, slow animation carefully designed to help you keep calm through a stressful process. But, the response came quickly.
Personally, I have measured direct correlations between our apps' latency&stutters vs revenue trends. This effect has been well publicized by Google and others. Users don't consciously give much of a shit. But, their behavior in end is significantly affected regardless.
>Sigh... Thinking up a different situation (or even an exception) doesn't make a general statement "not true". We aren't arguing mathematical theses here.
It's amazing how often people confuse discussion for mathematics...
While you're right that people sometimes value operations that take time, that's certainly an edge case - in general, snappiness leads to a better user experience.
Also: if an operation is fast, you always have the option to make it appear slower. The inverse is not true.
> in general, faster operations leads to a better user experience
That's just too general of a statement. That's like saying a landing page with a hero image is, generally, a better page. It's a good place to start, but it's just bad advice in the long-term.
I agree that response times from the backend should be fast, especially the initial response (as the article mentions). But anything on the frontend experience should be judged on a case-by-case basis.
Every time I've ever conducted user experience testing the results have always been far more positive the faster the app or page is. In fact, while it's still only anecdotal, I have never seen a decrease in any of the overall measurements of a user study where, between two versions, the speed of the app or page improved and little else changed.
Honestly I think it's a great general statement. I'm sure there are exceptions but that's the case with almost any general rule of thumb.
> That's like saying a landing page with a hero image is, generally, a better page.
No, not at all. Speed isn't everything and no one was suggesting that. A faster application, however, is always better than a slow one as long as you are comparing apples to apples.
With the way technology works nowadays it's actually very doable to get the 75th percentile to load really, really fast. But it's not without effort and earlier work may need to be redone, something many avoid at all costs.
If the UI is quick to inform I'm happy to wait for a big task to finish.
Even if the message is nonsense, I just want to know work is still occurring
"Reticulating splines" may not mean the app is just reticulating splines, but I know it's doing something, and I'm less concerne my Sim City hasn't loaded
But with things like my iPhone where the UI just halts after some requests and quits responding to input without me knowing why, is bullshit
I want my software to listen to me and inform me about the status of work I'm performing
I wish more work was done in operating system UIs informing users. I use a terminal a lot and can see what's going on as I work
The OS should be capable of detecting overload, and take control enough to alert to degraded general operation and inform as to what's up. Possibly offering options to resolution
I remember a study done a while back that found there are kind of "islands" of wait time that are acceptable, and "valleys" that are annoying.
I can't recall the numbers, but it was something like under xxxms was consitered "instant" and therefore wasn't an annoyance, from xxxms to 1 second is consitered annoying because it's too slow to be "instant" but not slow enough to let you context switch. Then from like 3-5 seconds was another "island" as it's enough time to context switch then return.
The end of the study found that by delaying some actions that were in the "valley" to be slightly longer and making them consistent it was reducing frustration while using the application.
I think there's a difference between "fast" and operations that take a while. Apps that do slow things should still be fast, even if fast just means "fast to pop up a progress meter".
Also, I ditched TurboTax for TaxACT in part because it was a much less ponderous experience. Anecdotes aren't data and all that, but I doubt TurboTax is super sophisticated about measuring their in app funnel.
The single site I visit the most when I am on mobile is Hacker News. The reason is simple: It's very fast and well structured, and most importantly it is free from bloat, no ads or unnecessary menus.
I feel like Hacker News is pretty much the best example for the concepts mentioned in the blog post and it proofs that beeing fast can be a sites main feature (or one of them).
In addition to this, it's important to validate on your own system. Your company's specific mix of traffic characteristics, offer, etc., means that not everyone will see the same results. Even for best practices.
I just ran a split test (>99% significance) where adding a heavy Salesforce chat widget to the site correlated with a 40% increase in the primary CTA actions taken. Adding in the chat leads, it was more like a 70% lift. From making the page load more slowly.
We verified that one a few times, but the results were consistent.
Your argument sounds like Stockholm-syndrome. Its not so much "faster" as it is latency. Any time the feedback between idea and execution starts to lag, it becomes painful.
Also, performance is important in the same sense that efficiency is important. If you can run the same logic with half the processing power you'll be able to either run more processes or shrink the necessary processor size.
Software may be a heavenly world of abstraction, but in the end the hardware has to pay for our sins.
I totally agree that app speed should be considered along side all other interface usability. There is probably a list of customer-requested features, bugs, marketing tricks, and other UI attributes that should be prioritized according to their ROI (projected affect on sales versus cost of implementation.)
Too often speed is seen as a back-end engineer thing when it should be handled the same way as any customer-facing feature.
It's a little thing, but 14.4kb/s should really be stated as kbit/s. 14.4kbit/s is 1.8kB/s. Looking at this reminded me how fascinating the Wikipedia entry for "modem" is [1]. Apparently "The first 9,600 bit/s modem was developed in 1968, and sold for more than $20,000."
I'm not sure I follow. If I can do what I want to do online and do it fast, then I run out of things to do much faster and I can get off the internet for a bit or do anything else.
You mean you've reached the end of the Internet? What's it like? :-)
Entertainment websites tend to keep providing more things to do, so you never run out. You can read them all day, like watching TV.
To help break the habit, It might be helpful to interrupt the temptation to read just one more thing, by making it cost a bit more. You don't want it to be effortless.
Unless you start spending more time online when you factor in the time spent waiting for page loads, plus potentially getting distracted and opening more tabs while waiting.
Totally off-topic but I always take slight offense when headlines talk about "apps" when the actual meaning is websites or web services.
For example I clicked on the headline expecting content about the speed of Android / iOS apps (which is something I care about professionally), but then was disappointed (and frustrated) to find that the actual content is all about web stuff.
That's really great feedback. I wrote something about API building a couple weeks ago without specifying I was referring to a _web_ API. I'll try to be much more conscious of this going forward. Thanks for the expression.
We're designed specifically for dynamic apps. In fact, we tell people to use traditional CDNs for static files. You can think of us as a combo load balancer and edge.
This means we do things like handle unlimited ssl certs (for apps that support custom hostnames), do load balancing on app instance, including geo load balancing, and even run code at the edge.
My favorite thing we do is app friendly caching. We're "aware" of app users and can handle caching logic well beyond the http cache control and vary headers.
I dug into this a little bit and can't figure out why. We don't actually do anything with JS that would affect scrolling. If you feel like emailing me with more details I'll keep lookin'.
For the vast majority of startups or websites, there should be 50 things on your to-do list that have the potential to increase your conversions more than 1-2%, and your time is almost certainly better spent working on those things than adopting a service like this. I know engineers like taking the same thing and making it faster, it is fun and satisfying, but it almost certainly isn't the best use of your time when it comes to the bottom line. And that is assuming making your site faster actually will increase conversions by any meaningful amount, which is a big if (very old and very specific studies be damned!).