That's always been the argument, but it's always been perfectly fine for me. I'm using it to run long running background tasks, or just let me switch between multiple remote hosts patching / debugging them, not to render some kind of HD game.
And yet... I've been using screen for 20 years and it never seemed inefficient, even when running on a 180MHz Pentium Pro. These days I have to believe whatever it's doing must be pretty minor compared to the overall system performance, but I'd be interested in hearing what it does so inefficiently.