This is a big factor in why Haskell is widely considered a really fun language to refactor in. The enforced purely functional paradigm makes adding nonlocal effects much more annoying, so later contributors can much more confidently reason about the state of their code (assuming you can find any later contributors in such a small community!).
I don't know whether the Lisps have a similar "fun to refactor" quality to them, since while they are in the functional camp, they're totally different and more flexible beasts architecturally.
IMO Lisps are not nice to refactor in that sense, but a dream to modify in general. In the case of Haskell and Rust, the compiler creates really tight and more importantly, global, feedback loops that have pretty good guarantees (obviously not perfect, but they're stronger than most everything else out there), while Lisp has basically no guarantees and forces you to execute code to know if will even work at all. This doesn't become apparent until you try to do large scale refactors on a codebase and suddenly you actually need good test coverage to have any confidence that the refactor went well vs in Rust or Haskell if it compiles there's already a pretty reasonable guarantee you didn't miss something.
Fortunately, Lisp codebases IME tend to be smaller and use small, unlikely-to-need-to-be-modified macros and functions, so large refactors may be less common.
I don't know whether the Lisps have a similar "fun to refactor" quality to them, since while they are in the functional camp, they're totally different and more flexible beasts architecturally.