> Streaming of computer screens is quite different compared to real life footage. UIs contain very shallow gradients (thus you want 10bit+), lots of flat areas which are easy to entropy-encode without any frequency domain quantization at all...
Well, you can get pretty good results with h.264 without B-frames. Modern composited desktops have caused traditional changed block (RLE, LZ) lossless codecs to perform badly.
> font rendering engines use subpixel anti-aliasing so you want 4:4:4 chroma
I think you can just gradually stream more (chroma) bits to stationary areas containing high detail (frequency) information, like text. In other words, tunable bit allocation between motion and stationary detail.
No matter what, there's no way to know what's the subpixel layout on the receiving end. Luckily Retina/HiDPI displays are increasingly lessening importance of subpixel rendering.
The thing is that 4:2:0 chroma is the default in most codecs because the main profiles only support that. You couldn't stream full chroma video on twitch even if you wanted to.
And subsampled chroma looks terrible on colored text, edges between flat colored areas and similar things.[0] So 4:4:4 is a must, only once you have that you can think about adaptive quantization for chroma.
I agree with that, but 4:4:4 also means there's twice as much data.
I think you should be able to get away with using chroma subsampling for moving portions while allocating bits in data stream for stationary areas, without subsampling (by using some heuristics, perhaps.)
Chroma from luma happens to work particularly well for video game content. It's one of the reasons Twitch is interested in using AV1 as soon as possible. Here's a talk by Thomas Daede where he discusses chroma from luma:
4:4:4 doesn’t mean twice the compressed bitrate, however. First, chroma residual already takes a minority of bits - luma residual and prediction info take way more bits and those are unchanged going to 4:4:4. Second, quantizing chroma more so there’s the same number of chroma bits as there would have been in 4:4:4 is approximately equivalent as subsampling chroma in the first place (okay, its actually slightly worse, so yeah you need maybe 1% greater bitrate to match quality with 4:2:0 with modern codecs on sequences for which 4:2:0 subsampling is transparent.)
Well, you can get pretty good results with h.264 without B-frames. Modern composited desktops have caused traditional changed block (RLE, LZ) lossless codecs to perform badly.
> font rendering engines use subpixel anti-aliasing so you want 4:4:4 chroma
I think you can just gradually stream more (chroma) bits to stationary areas containing high detail (frequency) information, like text. In other words, tunable bit allocation between motion and stationary detail.
No matter what, there's no way to know what's the subpixel layout on the receiving end. Luckily Retina/HiDPI displays are increasingly lessening importance of subpixel rendering.