I also mostly work in LaTeX but I draw with Asymptote: http://asymptote.sourceforge.net/gallery/. It matches the same bullet points as you have written but I like that is it standalone, not usually part of the LaTeX compile, and that it carries the MetaFont algorithms to 3D. There is a great tutorial at https://math.uchicago.edu/~cstaats/Charles_Staats_III/Notes_....

I use the graphtheory package a lot: https://github.com/taoari/asy-graphtheory.