If my understanding is correct, ESPHome need to be re-compiled and uploaded every time the config yaml is changed. Is it possible to separate the binary and the config so that for some config changes, there is no need to re-compile and upload the binary? Thanks.
I believe this is because the yaml is in fact the instructions for what to include in the binary. It wouldn't be feasible for the firmware to include all possible device and peripheral code and enable parts at run time.
I think you can see the esphome intermediate code generation in the file tree during compilation and see how the yaml sections map to blocks of C/C++ code being built.
In practice, this is absolutely no problem. It generally only needs to re-compile a file or two for small changes, which takes seconds, and the OTA update functionality works perfectly so you don't need to unplug it/bring it to your desk.