Many Poles have distinguished themselves in various fields of science and life. I recognize Madame Curie, Copernicus, Ulam, Pope John Paul II, Chopin, Polanski, and perhaps Wozniak. I had a Polish professor at Mellon University who talked a bit about it, and there was a lot.
Ulam was Jewish and there’s a difference. My grandfather was from the same area, Galicia and generation and he’d be damned if he’d self-identify as a Pole.
Your grandfather may harbor antipathy toward Poles, but he doesn't speak for Ulam. Ulam, like a number of Poles with Jewish roots, viewed himself as a Pole. Tarski is another example. Jan Brzechwa. Those in the Lvov-Warsaw school of mathematics, like Hugo Steinhaus. Etc.
The Poles have a long history of being a leading European nation in many areas. These "jokes" arose in the wake of Polish immigration to the US and the conquering of Poland by Nazi Germany and then the Soviets.
I am well aware of Polish genius in things like their early work on Enigma (as mentioned elsewhere). In the US (I don't know about other countries), 100% of Polish jokes are told without any inclination, on the part of the teller or the recipient, of actually believing that Poles are intellectually backward.
I have no idea how/why Poles became the subject matter of such jokes, but doubt that the history you outlined was the cause; after all, there are no such Czech jokes or Romanian jokes. I suspect that just as the jokes themselves are memes, at some point in the past Poles mysteriously ended up as the putative subject in some random fashion, akin to the arbitrariness of the ethnic group in this famous Austin Powers scene <https://www.youtube.com/watch?v=Zgp-0UxeBK0>, then spread as a meme.
PS - Steve Wozniak (also mentioned elsewhere) ran a Dial-a-Joke line featuring Polish jokes in the early 1970s.