FYI: The story title should be "Opensourcing Reclaim Protocol" not "How to make HTTPS verifiable"
> All data is routed through an HTTPS Proxy, which records the direction of data transfer without reading the encrypted content.
squints
> Certain parts of the request and response that are publicly known are revealed: For responses: HTTP response code and Date headers. For requests: Domain, Path, and Connection headers.
Um, domain (the Host header after establishing the connection), path, and headers (including Connection) are all encrypted. They claim this is public information but it's not.
I need more time to read into things but I'm skeptical thus far.
Howard Tracy Hall's descendant is my sister-in-law. They have lots of his stories in their family history, and she wrote a book for children about the process of inventing his process for the artificial diamond [0]. An uplifting and inspiring story that simplifies things for a child to understand.
I don't have access to the paper but based on the abstract and a quick scan of the presentation, I can confirm that I have seen results like this in Caddy, which enables HTTP/3 out of the box.
HTTP/3 implementations vary widely at the moment, and will likely take another decade to optimize to homogeneity. But even then, QUIC requires a lot of state management that TCP doesn't have to worry about (even in the kernel). There's a ton of processing involved with every UDP packet, and small MTUs, still engrained into many middle boxes and even end-user machines these days, don't make it any better.
So, yeah, as I felt about QUIC ... oh, about 6 years ago or so... HTTP/2 is actually really quite good enough for most use cases. The far reaches of the world and those without fast connections will benefit, but the majority of global transmissions will likely be best served with HTTP/2.
Intuitively, I consider each HTTP major version an increased order of magnitude in complexity. From 1 to 2 the main complexities are binary (that's debatable, since it's technically simpler from an encoding standpoint), compression, and streams; then with HTTP/3 there's _so, so much_ it does to make it work. It _can_ be faster -- that's proven -- but only when networks are slow.
TCP congestion control is its own worst enemy, but when networks aren't congested (and with the right algorithm)... guess what. It's fast! And the in-order packet transmissions (head-of-line blocking) makes endpoint code so much simpler and faster. It's no wonder TCP is faster these days when networks are fast.
I think servers should offer HTTP/3 but clients should be choosy when to use it, for the sake of their own experience/performance.
I see a few million daily page views: Memory usage has been down, latency has been down, network accounting (bandwidth) is about the same. Revenue (ads) is up.
> It _can_ be faster -- that's proven -- but only when networks are slow.
It can be faster in a situation that doesn't exist.
It sounds charitable to say something like "when networks are slow" -- but because everyone has had a slow network experience, they are going to think that QUIC would help them out, but real world slow network problems don't look like the ones that QUIC solves.
In the real world, QUIC wastes memory and money and increases latency on the average case. Maybe some Google engineers can come up with a clever heuristic involving TCP options or the RTT information to "switch on QUIC selectively" but honestly I wish they wouldn't bother, simply because I don't want to waste my time benchmarking another half-baked google fart.
The thing is, very few people who use "your website" are on slow, congested networks. The number of people who visit google on a slow, congested network (airport wifi, phones at conferences, etc) is way greater than that. This is a protocol to solve a google problem, not a general problem or even a general solution.
It's not. Think about what you search for on your mobile, while out or traveling, and what you search for on desktop/wifi. They are vastly different. Your traffic is not representative of the majority of searches.
I'm sure the majority of searches are for "google" and "facebook" and you're right in a way: I'm not interested in those users.
I'm only interested in searches that advertisers are interested in, but this is also where Google gets their revenue from, so we are aligned with which users we want to prioritise, so I do not understand who you possibly think QUIC is for if not Google's ad business?
Google sells that traffic to me: I'm buying ads from the Google Search Page and from YouTube and from ads Google puts anywhere else, so it is Google's traffic.
If google can't serve them an ad, they don't care about them, so QUIC isn't for them, so in order for QUIC to make business-sense to google it has to help them serve more ads.
The theory is if QUIC helped anyone out, Google (being huge) would be able to sell more ads to me if I use QUIC as well. In reality, they are able to sell me more ads when I turn off QUIC, which means this theory cannot be true.
It's strange to read this when you see articles like this[0] and see Lighthouse ranking better with it switched on. Nothing beats real world stats though. Could this be down to server/client implementation of HTTP2 or would you say its a fundamental implication of the design of the protocol?
Trying to make my sites load faster led me to experiment with QUIC and ultimately I didn't trust it enough to leave it on with the increase of complexity.
UDP is problematic because you can't recv() more than one packet at a time, so you get syscall limits that you could just ignore with a TCP-based protocol. There's a trick wasting lots of cores but it's bad for battery-powered devices. There's also iouring and af_xdp that look promising, but they aren't supported as widely as chrome.
HTTP2 I can't explain. HTTP2 should be better. I suspect it's probably an implementation bug because I can replicate lab performance and HTTP2 looks good to me in controlled tests.
I can always try turning it back on again in 6 months...
I don't know which site you're describing, but as a user living very far from major datacenters on an unstable and frequently slow network, HTTP[23] have been the best thing since sliced bread. How it looks in practice is getting hundreds of megabits/s when network is under low load (in the middle of the night, etc), but down to hundreds or even dozens of kilobytes/s in the evening. Sometimes with high packet loss. Always having pings of 100ms or higher (right now it's 120ms to the nearest google.com datacenter, 110ms to bing.com, 100ms to facebook.com, and 250ms to HN).
Back when HTTP2 was introduced and started becoming popular, I spent a few weekend hours on writing a short script to do a blind test of HTTP1.1 vs HTTP2 or some major sites where both were supported. H2 won every time, hands down. It was like comparing a 96kbit/s MP3 to a FLAC.
> UDP is problematic because you can't recv() more than one packet at a time
> but as a user living very far from major datacenters on an unstable and frequently slow network, HTTP[23] have been the best thing since sliced bread. How it looks in practice is getting hundreds of megabits/s when network is under low load ...
The surprise is that some users aren't able to access your site if you enable HTTP[23], and those that can will have worse latency on average.
There's a trick that I use to download large "files" -- I use content-range and multiple parallel fetch() calls. Because of the fast (lower latency) start of HTTP1.1 this outperforms H2 when you can get away with it. People who don't want to use JavaScript can use https://github.com/brunoborges/pcurl or something like it.
Cool. I don't think it helps H3 because the other end will just block anyway (because it's Linux only and the other side probably isn't Linux), and it seems to be a little slower than recvmsg() when there's only a small amount of traffic so I'm pretty sure it's not going to help me, but I'll keep poking around at it...
> It's strange to read this when you see articles like this[0] and see Lighthouse ranking better with it switched on.
I mean, Lighthouse is maintained by Google (IIRC), and I can believe they are going to give their own protocol bonus points.
> Could this be down to server/client implementation of HTTP2 or would you say its a fundamental implication of the design of the protocol?
For stable internet connections, you'll see http2 beat http3 around 95% of the time. It's the 95th+ percentile that really benefits from http3 on a stable connection.
If you have unstable connections, then http3 will win, hands down.
This is already how TLS offload is implemented for NICs that support it. The handshake isn't offloaded, only the data path. So essentially, the application performs the handshake, then it calls setsockopt to convert the TCP socket to a kTLS socket, then it passes the shared key, IV, etc. to the kTLS socket, and the OS's network stack passes those parameters to the NIC. From there, the NIC only handles the bulk encryption/decryption and record encapsulation/decapsulation. This approach keeps the drivers' offload implementations simple, while still allowing the application/OS to manage the session state.
Sure, similar mechanisms are available but for TCP ack offloading and TLS encryption/decryption offloading are distinct features. With QUIC there’s no separation which changes the threat model. Of course the root architectural problem is that this kind of stuff is part of the NIC instead of an “encryption accelerator” that can be requested to operate with a key ID on a RAM region and then the kernel only needs to give the keys to the SE (and potentially that’s where they even originate instead of ever living anywhere else)
Kernels enable IOMMU of the CPU, which limits the memory areas of the NIC can access to only to the memory it needs to access.
This is also why it should be safe to attach pcie over thunderbolt devices.
Although I think for Intel CPUs the mmunuded to be disabled for years because their iGPU driver could not work with it. I hope things have improved with the Xe GPUs.
I'd say Http1.1 is good enough for most people, especially with persistent connections. Http2 is an exponential leap in complexity, and burdensome/error-prone for clients to implement.
I was just setting up new phones for my parents last week. And seriously, y'all (making the software) have ruined computers.
Modern apps are so obnoxious and distracting and unusable it’s getting ridiculous. Tapping a verify link in an email would slide the email screen out, then slide in a browser, wait a second or two, then slide out the browser, then slide in the play store, then wait a moment, then slide in the app itself, which then had to update the screen after another second or two. Multiple screen changes flashing before my eyes made even me feel dizzy and confused. My mother had raised eyebrows and shook her head like she just got lost.
Then once you get in the apps, there's so many distractions that make the UI unusuable. "Hey did you know about this?" "Tap this to do this thing!" "Look over here!" not to mention various notification and permission prompts. The permission prompts are especially concerning because now there's so many of them to even start using a messaging app, for example, that you train yourself to just click through them instead of scrutinize them.
We've essentially gotten rid of text labels on icons, making it anyone's guess as to what the icon with the 3 colored circles does, or an icon with a shape, two lines, and another shape that I can't even describe succinctly. So many gestures make navigating the software a guessing game. My poor mother was like, "So to go to the last app I do this" only to gesture "back" in the same app, "Oh, I meant this", only to bring up the app selector. Nothing is discoverable anymore. Those little "learn how to use your phone" tutorials do nothing for you in "the real world." My father, who used to teach Commodore computers to his community, could no more figure out how to use a phone than his students could a Commodore.
We were in much better times when computers came with 100 page manuals and you had to go to a class or ask a techie how to use them.
I'm not even going to talk about the privacy problems in all this software.
I had to keep apologizing to my parents for my industry making the experience so bad and downright confusing.
>Then once you get in the apps, there's so many distractions that make the UI unusuable. "Hey did you know about this?" "Tap this to do this thing!" "Look over here!" not to mention various notification and permission prompts.
I feel a lot of modern UI/UX design isn't to serve the user, but rather the company itself. It's no surprise that it's unusable garbage because the user is merely a resource to exploit, not a human to deliver value to. See: Windows 11, smart TVs, and growth hacking.
Yes it's strange, outside of a couple of really great features (maps navigation is what comes to mind) my mobile phone is more and more something I avoid than seek to use and help me with the routine of my life.
At this point, 90% of what I do on my phone is probably text messages and maps. The rest of it is too inconsistent, changes too often, and is too demanding. I have to spend so much time turning off notifications and permissions for apps that I almost never bother installing new ones anymore.
> "Hey did you know about this?" "Tap this to do this thing!" "Look over here!" not to mention various notification and permission prompts.
This kind of thing is a pet peeve of mine. I think those things are attempts to mitigate some of the fallout from the fact that UIs no longer think discoverability is an important thing. So, instead of the UI itself incorporating discoverability, we're presented with these nearly-useless nags.
That is in part because of the users, because so many of them don't want to explore the UI or try to figure out how it works. Instead, they just want to know what they have to click on to do the thing that they want to do. I've tried to teach so many people how to use these things that just refuse to actually look around, and instead look at me helplessly instead of reading what the screen says. Bear in mind, that is when UI are made to be discoverable.
I just lump it in with all the other changes that signal that companies don't really want people to think about what they're doing on the devices anymore. They just want them to look where they want them to.
> because so many of them don't want to explore the UI or try to figure out how it works.
I've become one of them. It used to be that it was safe to experiment, because anything really damaging could be undone (which is an important part of discoverability). At some point, that stopped being a thing I could count on, so I've come to view experimentation as a risky thing.
But those nags don't solve the problem. They're really annoying, get in the way, and are rarely actually helpful. Nowadays, when they come up, I just close them without reading and get a bit irritated.
A huge problem with touch UIs is the lack of mouse-hover discoverability. At least with desktop applications that don't have labels on UI elements, usually a hover will raise a tooltip description.
Touchscreens are just incredibly restricted control interfaces in general
They're trying to replace physical buttons with actions like swipes and in my experience swipes, pinches, double taps, all that stuff only works some of the time
A right click on a physical mouse works every time
It boggles my mind that driver attentiveness nags are being required when driver assist features are _enabled_ instead of _disabled_. When you are most likely to commit manslaughter, the car (and the regulatory bodies making the policies) don't care if you're paying attention or not.
Yes all the lane keeping, auto braking, collision warning stuff could well be making some drivers feel safer trying to "send a quick text" or "just check my notifications" or (as we've seen in some "autopilot" accidents) reading a book.
Jurassic Park keeps coming to mind. It must be safe as long as you observe the signs, because it's an institutionalized park, right?
I don't know at what point I, as a first-time visitor to such a park, would start to question whether what I was witnessing was unusual or just part of the show.
Is there a way to query all DNS records? (I was surprised to learn that isn't the default.) This would be really helpful for troubleshooting people's Caddy questions (which are actually DNS problems).
However, each lookup happens serially right now, I'll take a look at making it concurrent per resolver atleast.
Edit: I just pushed the concurrent version of lookups in each resolver. Speed up is quite good around 70-80% on most domains. Will test this more before releasing to main!
I just pushed the concurrent version of lookups in each resolver. Speed up is quite good around 70-80% on most domains. Will test this more before releasing to main!
> All data is routed through an HTTPS Proxy, which records the direction of data transfer without reading the encrypted content.
squints
> Certain parts of the request and response that are publicly known are revealed: For responses: HTTP response code and Date headers. For requests: Domain, Path, and Connection headers.
Um, domain (the Host header after establishing the connection), path, and headers (including Connection) are all encrypted. They claim this is public information but it's not.
I need more time to read into things but I'm skeptical thus far.
reply