Do you give them "their" tooling? I can set up a project of the type you describe in about five seconds, because I have a template for it in my IDE. The best defaults for this that I've seen are provided by IntelliJ (do you provide this in interviews? It seems legally challenging to do) and would probably take me 5-10 minutes to navigate.
I think Java depends much more heavily on powerful tooling to do the heavy lifting, and my experience of using that tooling when I haven't had a chance to configure it in advance has been pretty miserable.
If they're coming in, I'll hand them a laptop with the project set up and ready to go, with IntelliJ running. If they're normally an Eclipse user I'll swap to Eclipse shortcuts and help them manage the IDE as we go.
If it's remote, I'll ask them to share their screen with me with a project set up and ready to go, having emailed them a copy of the interface we're going to implement about 30 minutes before the interview.
I think Java depends much more heavily on powerful tooling to do the heavy lifting, and my experience of using that tooling when I haven't had a chance to configure it in advance has been pretty miserable.