It's a bit rough around the edges, but I made precisely this. You bring your own CalDAV server, and it will create a REST API that allows for scheduling. Alternatively, the CalDAV functionality is separated into a library, so you don't have to use the provided REST endpoints.

The front-end is separate, and I haven't put much love into making it useful for other people. But it does work.

https://github.com/justinrubek/calendar-scheduler