It's just a informal term for the things mathematicians study. So the natural numbers are mathematical objects, as well as a set, a function, a manifold, a vector space, etc.
This is actually a surprisingly complex question in mathematical philosophy -- I unfortunately can't find a copy online right now, but if the idea of mathematical objects interests you I'd highly recommend Paul Benacerraf's 1965 paper "What Numbers Cannot Be", which explores the complexity of the idea of even a single integer as an "object".
For those interested, the article is actually called "What Numbers Could not Be" (many references have "What Numbers Cannot Be" so maybe it was published under both names at some point?)
Thank you! I just checked my copy of Cambridge's Philosophy of Mathematics and it is indeed "could not" -- the PDF I have on my laptop, however, is "cannot". I wonder when/where that happened.