I didn't say it was rude, though. I would have hoped he could've just said "sorry, but I'm too busy to talk right now."

Like I said, it was likely wrong place/wrong time. First impression doesn't mean all subsequent impressions.

Anyway, I hope I do get the chance to meet him eventually when it's the right time.