As others have pointed out, search engines also need to parse your pages to extract text and links. You can't really test if Google parses malformed HTML the way you think it ought, so I would say better safe (valid) than sorry.