I believe it, but I mostly hear about how user retention drops off if the website takes more than 1s to load.

And while I believe other types of slowness make users unhappy without seeing the literature, the people driving our priorities don't see the $ impact at all

I hate slow sites and am pretty aggressive about bailing on anything that I consider slow.

But speed is a visitor's feature, not a benefit. The benefit (to the web site operator) of an ecommerce site is $ spent by visitor.

So for example yes, the Amazon site feels slow and bloated, but they've been doing live A/B testing for over 20 years and I'm pretty sure they don't care about speed unless it affects conversion; they'll be happy to cram something in even if it slows the site down, if it increases revenue.

Amazon seems pretty darn fast to me. You click and stuff happens. It's faster than Twitter or Github (the first two sites I compared it with).