If you happen to be a quiet place it will likely be easier to concentrate without the rain sounds. "Easier" in this sense is a bit relative, however. An environment with fewer external distractions will lead to a selection effect where you "catch" yourself more frequently and return back to the breath. Thus, you may feel like you're not concentrating well, since you "keep getting distracted." It's kind of like the Dunning-Kruger effect of the beginning meditation :)

On the other hand I find music and background conversations especially distracting and like to listen to pink-noise or other non-varying noise sounds to make concentrating easier.