You're right! You need a strong engineering culture around its use to avoid its pitfalls, and without that, the results are often regrettable. And I agree, too, that this is a failing of the standard, not of those who use it.
But it's also true that JWT has legitimate use cases; I'm working with it right now in my professional capacity, and we are using it precisely because "twenty bytes of hexified /dev/urandom" isn't a terribly efficient option at our scale - we could do it, but we won't, because it doesn't add value to the business.
As you say, though, the crypto sucks. You know a great deal, both in the absolute and relative to my own knowledge, about such things. What, in your estimation, is the means most likely to produce an outcome where we have something of roughly similar utility and ease of use, but with solid crypto and few or no bear traps for the unwary engineer? (Beyond just removing the algorithm negotiation, or at least the "none" option.) Or does such a thing exist and I'm just one of today's lucky 10,000?
You can start with running your JSON data (or whatever serialization format that you choose) through NaCl/libsodium's crypto_secretbox() and you already get more than what JWT offers by default.
It still doesn't include everything you need of course: at the very least you'll still need to implement expiry validation and some token revocation mechanism. But you actually get more value out of the box than you get with a JWT library: your token content is encrypted, your using a solid crypto library and you don't have to mess around with any knob to secure your implementation.
It doesn't offer support for hot-swapping crypto algorithms, but for what it's worth, cryptographic agility is the root cause of many issues we had with TLS in the past[1].
If you end up needing both symmetric and asymmetric cryptography in your tokens, you better treat them as two different types of tokens, because they usually are. I think tptacek already said, but asymmetric crypto is rarely interchangeable with symmetric crypto - you usually use them in very different cases.
Rails applications have been doing stateless auth in encrypted cookies for over a decade, and the constructions they use to do it are safer than JWT's, and orders of magnitude less complex. I would want to push back on the idea that a good use case for JWT is "we need to do stateless auth".
Stateless auth is a bad place to start. But, when you need it, there are better ways to do it than JWT.
But it's also true that JWT has legitimate use cases; I'm working with it right now in my professional capacity, and we are using it precisely because "twenty bytes of hexified /dev/urandom" isn't a terribly efficient option at our scale - we could do it, but we won't, because it doesn't add value to the business.
As you say, though, the crypto sucks. You know a great deal, both in the absolute and relative to my own knowledge, about such things. What, in your estimation, is the means most likely to produce an outcome where we have something of roughly similar utility and ease of use, but with solid crypto and few or no bear traps for the unwary engineer? (Beyond just removing the algorithm negotiation, or at least the "none" option.) Or does such a thing exist and I'm just one of today's lucky 10,000?