I think you've struck on the actual reason: Rust programmers don't perceive compile times as slow, and don't really view it as a problem. Thus, nobody works on making them faster.
Every language has tradeoffs, and every language community has priorities. In general, the Rust community doesn't care about compilation speed. For now, the community has basically decided that incremental cached compilations are good enough.
Which is fair, because there's only so many engineering hours, and the language has a lot of other priorities that fast to compile languages like Go ignore.
I'm biased towards C and Go's way of thinking about language design, which I know a lot of other people hate. But, there's also the universal problem that once you introduce a feature into a language, people will have a field day using it in contexts where it's not needed. Just like Perl programmers have never met a regex they've never disliked, and C++ programming have never heard of a bad operator overload, Rust programmer's have never seen a bad procedural macro or external crate dependency. Showing just a little bit of restraint using complex or slow to compile language features goes a long way, but it seems like most devs (in all languages) can't resist. Go is partially fast to compile because it just tells devs they aren't allowed to do 90% of the slow-to-compile things that they want to do.
Powerful languages like Rust and C++ give devs the choice, and they almost always choose the slow-to-compile but elegant option. At least, until the build hits an hour, then they wish compile times were faster. For the record, I'm not bashing C++ or Rust, I'm a C++ developer by trade.
Every language has tradeoffs, and every language community has priorities. In general, the Rust community doesn't care about compilation speed. For now, the community has basically decided that incremental cached compilations are good enough.
Which is fair, because there's only so many engineering hours, and the language has a lot of other priorities that fast to compile languages like Go ignore.
I'm biased towards C and Go's way of thinking about language design, which I know a lot of other people hate. But, there's also the universal problem that once you introduce a feature into a language, people will have a field day using it in contexts where it's not needed. Just like Perl programmers have never met a regex they've never disliked, and C++ programming have never heard of a bad operator overload, Rust programmer's have never seen a bad procedural macro or external crate dependency. Showing just a little bit of restraint using complex or slow to compile language features goes a long way, but it seems like most devs (in all languages) can't resist. Go is partially fast to compile because it just tells devs they aren't allowed to do 90% of the slow-to-compile things that they want to do.
Powerful languages like Rust and C++ give devs the choice, and they almost always choose the slow-to-compile but elegant option. At least, until the build hits an hour, then they wish compile times were faster. For the record, I'm not bashing C++ or Rust, I'm a C++ developer by trade.