> Is there any evidence that French has some unique qualities that render it unusually difficult to understand when not spoken by a someone fluent in the language?

Usually it’s just English exceptionalism and the centuries-old rivalry between both countries. Don’t expect much objectivity from either side. From Americans, it’s usually ignorance of a foreign language. Spanish-speaking Americans don’t tend to say this sort of thing.

> I’m reasonably competent in French and if there is such a feature, I don’t see it.

There is none.