How does an upscaler in your TV work? How does it create information out of nothing?
Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image.
It works amazingly well, and looks like magic.
Maybe there was no visibility of pores on the face in the 480p downscale, but your model can learn to reproduce them faithfully.
Sony has access to billions of movie frames in extremely large resolutions. Their engineers are definitely using this large amount of information to create statistical filters which upscale your non-hd, or maybe your HD to 4k HD. These filters work better than deterministic methods in this article. Why? Because the filters know much more about the distribution of the source (distribution of values of each individual pixel). They have exact information that one instead tries to assume (author in the article assumed that something in the source - be it noise or something else - behaves according the to Gaussian). If you know how to find the proper distribution, instead of assuming it, you can move closer to the information theoretical limits.
Just imagine how fast these filters can be if you put them on an FPGA, it also explains why TV sometimes cost more than $2k.
If you knew that your images would only contain car registration plates, you could definitely learn a filter that would be very precise in reconstructing the image when zoomed, you'd now find CSI zooming a little bit more realistic :D
> you could definitely learn a filter that would be very precise in reconstructing the image when zoomed
Yes, your result would be a very clear image of one possible license plate. An algorithm may be able to do slightly better than a squinting human, but ultimately you can't retrieve destroyed information.
Of course you can't retrieve destroyed information.
But information is not destroyed by perfectly unpredictable (uniform) distributions of noise.
> Yes, your result would be a very clear image of one possible license plate.
Fortunately there are methods of evaluating how well your statistical filter works, if it's precise enough you'd be fine with indeterministic nature of your filter. Or even better, you could generate all of the highly probable licence plates, instead of having only one - given by your deterministic algorithm.
From what we know about the software on these TVs, upscaling there works using whatever code the cut-rate programmer could 1) google and 2) efficiently integrate into the product with minimal fuss.
Upscaling is a very state of the art technology, not some layman's solution.
There are firms that specifically targeted upscaling as their product and made millions with their state-of-the-art tech. Currently upscaling is in the rise again with 4K TVs. Back in the days they made some incredible chip solutions, sold them expensively to Sony, Samsung and similar. Sony realized they can, with all of their resources (super-HD movie database) make incredible upscalers.
Just imagine that Sony has stored whole movie The Walk (distributed by Sony Pictures) in your TV in 4K resolution, the moment this movie is displayed on your screen through some lower resolution sources, they find it in the database and display the 4K content. Of course, that's highly inneficient and memory intensive, thus, they use statistical models to efficiently store movie material and have fast chips to quickly approximate the real upscale.
This will then, if the sample (number of movies) is high enough, work well on all of the movie content.
Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image.
It works amazingly well, and looks like magic.
Maybe there was no visibility of pores on the face in the 480p downscale, but your model can learn to reproduce them faithfully.
Sony has access to billions of movie frames in extremely large resolutions. Their engineers are definitely using this large amount of information to create statistical filters which upscale your non-hd, or maybe your HD to 4k HD. These filters work better than deterministic methods in this article. Why? Because the filters know much more about the distribution of the source (distribution of values of each individual pixel). They have exact information that one instead tries to assume (author in the article assumed that something in the source - be it noise or something else - behaves according the to Gaussian). If you know how to find the proper distribution, instead of assuming it, you can move closer to the information theoretical limits.
Just imagine how fast these filters can be if you put them on an FPGA, it also explains why TV sometimes cost more than $2k.
If you knew that your images would only contain car registration plates, you could definitely learn a filter that would be very precise in reconstructing the image when zoomed, you'd now find CSI zooming a little bit more realistic :D