I wish I had something useful to say, but that sounds honestly amazing. If you could do wifi at the same time, then that'd be a perfect email/ssh/irc device.
i think that the esp32 is probably a decent microcontroller to use for this purpose, in place of the pi zero; it does have a decent sleep mode and also wifi
its energy efficiency per instruction is not the best but is certainly acceptable