You've just pretty much described the motivation for core.matrix: it's an API that wraps various other back end vector/matrix libraries (including JBlas etc.) with a nice, standard Clojure API.
Yup. I'm really glad to see that you guys are working with Incanter now as well.
BTW, as a total aside, I'm a big fan of Clisk. I think you nailed the API for functional image manipulation. I'm interested in seeing an OpenCL backend for it. If I get anything working well, I'll be sure to contact you.