It can bog down depending on your communication patterns. Context switching isn't too bad, it's 'green threads' so there's not the OS overhead of switch threads when you switch between Erlang processes; it's not zero overhead of course. Messaging passing latency is mostly driven by memory latency and lock acquisition; if you're in a big system, message queue locks tend to be uncontested as there's enough things going on that it's unusual for two processes to message the same third process at around the same time. But you can't really get away from that --- regardless of your language, if you've got multiple threads of execution working on the same data, you've got contention and that'll increase latency; even lockless algorithms have contention on the atomic values, and you have memory communication between processors.

All that said, BEAM isn't chosen because of its high performance, it's chosen because it enables a simple programming model for distributed systems while being fast enough. It's gotten faster over time too; of course, so has everything else.

It's all green threads, so you don't have real context switches and the messages don't have to be passed between (real) processes. Of course there are downsides to that kind of approach: good luck calling a C library function that you have to pass a callback into, anything else where you need precise control over your OS-level threading is difficult (and while BEAM has a good reputation, there's always the risk of e.g. priority inversions happening), and if the VM scheduler doesn't handle your workload well then you're limited to whatever control they give you.