Sometime over the past several years we seem to have discarded any specific definition of the word "fascist". All we're left with is just "somebody whose politics I disagree with".
I don't believe Heinlein was a fascist at all, but he's hard to pigeon hole. He was a radical libertarian, yet believed that a benign military world government was possible and even desirable. Well, a military world government sounds petty close to fascism, but he somehow thought that this would be the best way to guarantee individual liberties.
So his goals were libertarian, but I can't help thinking his intended means to achieve them would wind up, er, not doing so at all even slightly.
I think the backlash to describing Heinlein as "fascist" says more about the average person's understanding of real-life fascism than anything. I get the impression most people just replace the word with "evil" in their head rather than considering it on its merits and understanding what his thoughts on it were. There's often an implicit unstated argument there, like "X author is fascist, fascism is evil, therefore X author is evil, therefore you should not read their books", which seems a little more extreme than is warranted.
On that note, Heinlein wrote all sorts of weird, contrarian themes into his books. Many of them pushed against the prevailing social norms and beliefs of the time, from sexual norms to ethics to race relations. Trying to pin down his actual beliefs is difficult because he would often write one book taking one stance on an issue, and another later with a totally contrary stance. Often when I read Heinlein I find myself strongly disagreeing with what he's putting down, but fascinated with the question or angle being posed/presented.
You and I (as an uncle reply) overlapped in flight. I agree with what you're saying, but I think that even before getting to that, the word "fascism" has lost any precise definition. As far as I can tell, it can no longer be interpreted to mean anything more specific than "politics that I disagree with".
And then, of course, your comment about people being unwilling to even consider the thoughts of those they disagree with steps in.