I have dyscalculia (and dyslexia) but I can keep time like you wouldn't believe. I once met a hotel lobby musician who was unreal and I said to him, you must be excellent at math, he said actually I can't even do basic algebra. I have a very rich audio/visual memory and thinking style, I can think in sound and movies, but that thinking style kinda creates the limiting factor for numbers and letters (literal no framework for them) maybe?
After spending a huge chunk of my time growing up in rhythm sections as the bass player, not only can I follow time like nobody's business, but I've grown to develop a penchant for weird time. Give me a time signature beyond 3/4 or 4/4 (15/8, anyone!?), or more importantly, give me a musician who will play some heavily syncopated rhythms and polyrhythms (looking at you, Chris Dave) and just do their best to stretch, bend, re-shape, and - pardon my french - fuck time. That's my jam.
Ask me to do anything beyond add/subtract/divide (forget algebra entirely lol), though, and my brain shits itself. Even at a certain point, division becomes complicated for me. But I will hit that pocket and never, ever lose the beat every single time, and that's always been interesting to me.
If you're in the pocket anything you play will be on time.
Pretty much anyone can stay in a 4/4 time if they have a lick of musical ability.
With the weird time signatures like 13/8 or 7/4 it can be tough to find the pocket, so in those cases the vibe tends to divide into 2 sections that fit right at first (like one section of 7/8 and one of 6/8, or one section of 4/4 and one of 3/4) and you bounce between them until you sew them both together and then you can stay in the pocket for all of them.
It's harder to explain via text than it is in music practice.
And with enough practice, "weird" becomes ordinary. I can keep rhythm in 7/8 and 7/4 almost as easily and automatically as 4/4 now, but 10 years ago it still took work. I sometimes go entire days with unbroken 7/8 music going in my head.