Being a non parametric model, they are pretty garbage for generalizations outside of the domain of the data they have "seen" and they are also not very data efficient. One area where I have seen success for them is when they are applied to automatic hyper-parameter selection for neural networks.
It's the opposite, they are very data efficient (compared to neural networks). Also, they are a bayesian method and give some mathematical guarantees. They are computationally expensive though. You can use them in neural nets to do few shot learning [0].
