Looking at page timing, people seem less jarred by the slight contrast difference, even if they would have a hard time distinguishing the two. As for why, I am not sure.
I'd love to see a blog post on some of the nitty-gritty details on this. Even if there's no known root cause, it'd be fascinating to see your analysis on this.
If you think about paper, it's not perfectly white because of the material and light passing through it and reflecting off it. Neither is the ink perfectly black. I've also found that a mix of shades is preferable to either pure white or black. On my latest site I used:
