It's not just OSI. Debian, FSF, etc. all don't see that as free software specifically because it restricts the ways that you can use the software - thereby defeating the point of having the source code in the first place.
That's no different than some EULA saying you can't do xyz with this software without purchasing a $xxx,xxx license from the vendor or whatever: it's the same effect. It's an artificial boundary put in place that restricts the user from doing something.
I don't find that particular view "esoteric" and I'm curious about why you do.
I don't hold to the extremes OSI does as to whether something fits their esoteric view of "open source", however