I think the best clarification of this is to separate out fault from responsibility; and the analogy to explain was snow on your sidewalk. Is it your fault that there's a foot of snow and ice on the walkways in front of your house? No -- you had no control over that. Is it your responsibility to shovel it / put out salt to prevent people from falling and hurting themselves? Yes.

Similarly, if you're white and/or well-off, is it your fault that things are tilted your way? No. Is it your responsibility to try to do something to make the field more level? Yes.