I love Thunderbird. I use Thunderbird. I want Thunderbird to survive and be successful. But I've never actually been able to make full use of things like flags, smart folders, or the built-in anti-spam because it's all local to a single computer. Those benefits don't follow me to my phone, or to a friend's house, or between my laptop and desktop.
We either need more extensions to IMAP (but what clients would support them?) or we need better web apps.
Not being able to teach the server about spam is really a big shortcoming. I have it on my todo list for years now... Apparently there even exist standard protocols for that, perhaps there are already plugins for TB that can do that? And of course the server has to play along, too.
I went a bit off track, but I did want to make a relevant point: For my use case, the features you enumerated aren't competitive advantages because they're locked away in a single client on a single machine. Thus, for me, there are entire classes of features where native clients can't innovate, because I can't rely on those features being available in more than one place.
So from my perspective, Mail.app isn't "way behind" Thunderbird. Rather, they're at parity, and Thunderbird has a lot of other stuff that I can't use.
It won't help you with your phone, but isn't there a version of thunderbird on portableapps.com? These install and keep their data on a USB stick, so you can take it with you between desktop, laptop, work and friends' computers (no install needed).
We either need more extensions to IMAP (but what clients would support them?) or we need better web apps.