I might be concerned if the only thing the random Glass wearer recorded was you walking your babies, but if that's a small part of his daily record, so what? Without Glass he would have seen you just the same.
Plus the panopticon of everyone else with access to the data. I think we'll be able to broadcast our recordings publicly, and if so then tools will exist to take advantage of all the data in the public broadcasts and then we'll all be able to look into each other's lives in an equal way. People who don't publicly broadcast may well be shunned.