If every site used the same font, maybe. But different font faces vary in readability at the same size. Also, we still have to set sizes for things like titles, subtitles and other elements to create a hierarchy on the page.

'Don't resize my fonts' is such a narrow view of whole range of design considerations.