I assume you mean located very close to the rotation axis rather than at some other latitude? I think it's convenience, because you can certainly do this at other points close to the pole, but it would involve more slewing I guess and it's mechanically simpler because you don't need to do much field rotation. At the pole, you can practically leave the altitude fixed and only move in azimuth to track something.
There are also telescopes in other places that are fixed in place, they stare at some angle and let the Earth's rotation do the work for surveying.
There are also telescopes in other places that are fixed in place, they stare at some angle and let the Earth's rotation do the work for surveying.