Markdown has about the same feature set of Microsoft's WordPad (the bulit in RTF editor in Windows). If the document you're writing can be structured with simple Markdown, why not just use an RTF editor and enjoy WYSIWYG, graphical image resizing and positioning, and great portability?
I have tried to see the use case for Markdown for a long time but have been unable to do so. And yes, pandoc understands RTF.
I think the best use case for Markdown is when you specifically want 'not too rich text' – for example, I like the way it's used on Stack Exchange. It needs to be combined with another tool, like a web application, to take care of layout.
I use it because I can easily insert Latex into it for math. I also use it to generate automated reports (e.g. update this document you made last year with new data).
I have tried to see the use case for Markdown for a long time but have been unable to do so. And yes, pandoc understands RTF.