ProcessPoolExecutor doesn't let you supply a callback to run on worker process exit, only startup. Perhaps I could've looked for and tried something like atexit (https://docs.python.org/3/library/atexit.html)? In any case I don't want to touch my code at the moment until I regain interest or hear of resource exhaustion, since "it works".
Fair enough. If you ever do decide to touch it to address that, I suggest (in preference order):
- Subclassing ProcessPoolExecutor such that it spawns multiprocessing.Process objects whose runloop function wraps the stdlib "_process_worker" function in a try/finally which runs your at-shutdown logic. That'll be as reliable as any try/finally (e.g. SIGKILL and certain interpreter faults can bypass it).
- Writing custom destructors of objects in your call arguments which are aware of and can do appropriate cleanup actions for associated SharedMemory objects. This is less preferred than subclassing because of the usual issues with custom destructors: no real exception handling, and objects sneaking out into long-lived/global caches can cause destructors to run late (after the interpreter has torn down things your cleanup logic needs) or not at all.
- Atexit, as you suggest. This is least-preferred because the execution context of atexit code is .... weird, to say the least. Much like a signal handler or pthread_atfork callback, it's not a place that I'd put code that does complicated I/O or depends on the rest of the interpreter being in ordinary conditions.
