I can't speak to other materials but aluminum recycling is incredibly less energy intensive than refining from ore. It takes an incredible amount of energy to convert aluminum oxide into aluminum and not much to melt aluminum once it's in that state.
The same goes for glass, although the energy difference isn't as big as with aluminium. The main problem with glass is that coloured glass can't easily be made into clear glass, so it gets used for making green/brown glass. Glass is also very suitable for reuse, so it doesn't necessarily have to be recycled unless the glass item has broken.
Sure, yes, clean glass and aluminum are often easy cases. I don't think there is that much trouble with compost either?
The problem seems to be with many of the other things that are allegedly recyclable. Just because someone somewhere can recycle it doesn't mean your local recycling facility can do anything with it.
Another insanity is that companies that produce certain waste are required to offer to recycle it. That's fair in most circumstances, in others, not so much.
I live in Hawaii, and I usually buy local eggs. They come in a polystyrene container, and polystyrene is something that shouldn't end up in landfills. So the company graciously offers to recycle the container for me, the only thing I have to do is mail it to Washington State. From Hawaii.
(Thankfully, Hawaii incinerates most trash, so that goes straight in the trash!)
The polystyrene containers are clearly in the minority of the brands that I can buy here, it's just that the ones I prefer come in it. If you buy eggs that come in cardboard at my local Safeway they're usually mainland eggs.
It would be interesting to know if the environmental impact of the plastic containers for the local eggs offset the environmental impact of transporting mainland eggs to Hawaii.
Hawaii has a high percentage of oil-powered electricity, and it burns all trash, so me throwing the plastic container in the trash should offset some oil, which drastically reduces the impact of the plastic container per se. So the polystyrene containers might be the better option.
Figuring out the actual best environmental option is really really hard. And the exercise above is just for god damn egg cartons.
Then again, there's plenty of low-hanging fruit to solve before my choice of eggs start to count. If you live in a place that has landfills, that should be the number one priority to fix. Landfills are terrible, burn the trash instead, it allows you to recover a lot of energy.
My best guess is that Hawaii doesn't have a wood pulp industry at all, so cardboard containers have to be imported. But Hawaii imports a ton of oil already, so having a local factory that makes plastic packaging out of oil is probably cheaper.