The 24-96 is different master, some sound engineer just had a field day in the studio and produced a better mix. Repeat the test with a 16-44.1 version downsampled (use something like sox with the ultra high quality resmapler) from the 24-96 version and I guarantee you will not be able to spot any difference compared to the "true" 24-96 version.
I understand the theory of this, but cannot reconcile it with an experience I had in person a decade ago.
In the acoustically prepped monitoring booth of his recording studio, a friend of mine tried to give me an ABX test of 24bit 96 kHz recording and its 16bit 44.1 kHz rendering that was supposedly done right. I heard the difference and easily picked the high-rate one that sounded more life-like. With my best effort, I described it as having a clearer high frequency spectrum, while the other sounded muffled in comparison.
I am left wondering if the 44.1 kHz file wasn't actually rendered correctly with dithering, or if my friend failed to actually get his studio equipment to play it back correctly. I.e. was some overly aggressive low-pass filter done during the conversion or during the playback.
As you said yourself, it was most likely a rendering issue. A bad low-pass filter would have attenuated the high-end when converting to 44khz. Also, this is afaik the reason why all modern audio uses 48khz, you get a little bit more head-room when designing a low pass filter and you can even choose a less aggressive and perhaps less computationally expensive one that still won't have an effect on the humanly perceptible frequencies.
I think the reason a lot of modern audio is at 48k has more to do with it being accompanied by video, which has independently settled on sampling rates of 48k, 96k, etc.
