CDNs could whitelist HTTP header keys, constrain HTTP header values, and regenerate a full clean header to the web server as part of their security services.
CDNs already parse headers, and the CDN developers have the knowledge and know how to correctly constrain header values.
Well this is the thing I'm confused about, having worked on some cloudfront stuff last month, documents like https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope... gave me the clear impression that every header cloudfront passes to the origin will be considered a cache key. In particular if you configure it to "forward all headers", which is what would surely be needed to pass this custom "X-Oversized-Header" onwards, it would effectively disable all caching.
I'm clearly going to have to spend another day staring at the cloudfront documentation.
And why weren’t all headers considered in the cache key to begin with? The key computation will never not be O(1).
Only thing I can think of is too many cache entries with the same cache values for the case where requests with different headers produce the same API response.
CDNs already parse headers, and the CDN developers have the knowledge and know how to correctly constrain header values.
There are 100's of different HTTP Header attacks e.g. I like this one that returns a response from a different persons HTTP response into your HTTP response: https://portswigger.net/blog/http-desync-attacks-request-smu...