defer is best compared to try/finally or unwind-protect and friends. You can defer any action, not just whatever the IDisposable happens to cleanup. It also gets access to the lexical scope for its deferred actions.

You could imitate that with IDisposable, but it would be overkill compared to just using finally (in C#).