Giving your friend the benefit of the doubt, sometimes one gets the wrong perspective from a careful, qualified statement.
W. Sidney Allen wrote Vox Graeca[1] and Vox Latina which are well-known works which detail both what we know about pronunciation as well as explaining how we came to know these things. We surely don't know everything and scholarship evolves, but it is hardly like what you seem to be suggesting.
It is interesting to know about some of the sources: ancient grammarians wrote about pronunciation, people poked fun at how other people talked in letters, epigraphers sometimes made mistakes on the order of "could of".
