I worked with GiNaC https://www.ginac.de/ and noticed that the bottleneck was variable substitutions, e.g., replace a variable with a constant and simplify the expression. I'm still dreaming of an efficient CAS.
https://github.com/dsharlet/ComputerAlgebra (disclaimer: my project) might do what you want, it lets you manipulate expressions and then compile them to "native" .net functions that can be evaluated with high performance.
Thanks! I'll have a look at it for sure. During my phd I developed a tool for reachability analysis of polynomial dynamical systems https://github.com/dreossi/sapo I needed expression manipulation to compute Bernestein coefficients of polynomials