The new breed of garbage collectors may change this significantly for the better. They need to be explicitly enabled, but there is a GC (G1) that may be better suited for desktop applications (low latency pauses, relinquishes memory to the OS more frequently while idle, etc.) available in Java 11.
Thank Garbage Collected languages. I just checked the IDEA config and found its maximum heap is set to 750 MB by default. You typically wouldn't need 32GB to run it.
I changed the maximum heap for my IntelliJ IDEA instances to 2GB and haven't seen a slowdown since. 750M is too low if you're working on a large project imo. Adding a dependency to my pom and immediately debugging used to cause the IDE to chug while it scanned everything.