Mostly different in terms of performance and wear on the drives. If you rsync over and over, it has to scan basically the whole filesystem for changes each time. Zfs snapshots don't. The snapshot is ~instant and the calculation of what to send has no need to examine any files.
I don't think the performance and drive-lifetime hit of running rsync every 10 minutes would be good.
Zfs should have an edge in terms of atomicity as well, but in practice I'm not sure how much that matters. I _think_ it does matter but isn't perfect (zfs can't trick applications into doing atomic writes if they're not already, but it won't have _another_ worse layer of breaking atomicity like rsync must).
Depends what else the machines are doing, and how much ram, and the rsync settings.
If you read all the files on both sides, every time and don't have more ram than disk, it's going to be a lot of work. If you're just looking at directory entries most of the time, there's a good chance that's all cached and it's no disk load, other than the small changes.
I ageee with you though that atomicity is a big difference, if it matters, and in most cases, it probably doesn't.
Personally, I've mostly stopped doing rsync backups in favor of zfs send, but I've still got one I need to get around to changing. Sanoid/syncoid is pretty decent for less effort snapshotting and syncing snapshots; but I haven't done anything with encrypted datasets. For most of my systems, I'd prefer recovery over security. For the one system in iffy hosting, it runs full disk encryption as a layer below zfs, so it's zfs sends are cleartext, too. (The hosting facility has given me other customer's unwiped disks; better for me to assume my disks won't be wiped)
> If you read all the files on both sides, every time and don't have more ram than disk, it's going to be a lot of work. If you're just looking at directory entries most of the time, there's a good chance that's all cached and it's no disk load, other than smthe dmall changes.
Yeah that's a good point. I know that rsync is _quite_ clever, but at least any incantations I've ever done it still hits the drives a good amount. I'd ballpark guess a couple of orders of magnitude better than just "cp -r" or something, but still a couple of orders of magnitude worse than zfs snapshots.
Yeah you're 100% right it'll depend on bunch of variables though.
> Sanoid/syncoid is pretty decent for less effor snapshotting and syncing snapshots; but I haven't done anything with encrypted datasets.
I'm not sure I'd recommend it, but I use both directly on encrypted datasets. I have tested recovery a couple times and it works fine, but I've read some cautionary tales too. I _think_ they're all old issues?
Regarding atomicity, do you rsync from the live filesystem or from the hidden .zfs/snapshot dir?
My impetus for swapping to a send-based approach from an rsync-based approach of running an incremental (just from the normal non-snapshot filesystem view) then taking a snap on the far side was encountering corrupted encrypted containers. If rsync ran while a container was mounted and being written to, it'd get an inconsistent view of the underlying file and produce a nonsense diff, resulting in an unmountable container in the backup. This doesn't happen with send because it has a consistent view of the blocks it needs to replicate, but as I was writing this I realized that running rsync from zfs's view of the snapshot might get around that. Of course, at that point it's probably easiest to just use send.
My rsync backup goes from the live filesystem (and then does some snapshot-like things on the other end with hard links and what not), but it's for my househould's shared network drive, so atomicity isn't super important. Mostly there's not many changes, and if there are, it's ok if it takes a couple snapshots to settle.
I didn't originally have that area as its own zfs filesystem, and it wasn't even originally on zfs, but I moved things around when setting up a new, offsite, backup system... Just haven't gotten around to redoing the old backups. I don't think I'd spend any more time on rsync based backups, given how I use things now; incremental zfs means no need to compare, which makes me have good feelings.
For SSDs I'm not sure. My data drives, SSDs are still too expensive so I haven't switched. Spinning rust it definitely does matter (for drive lifetime and also contention).
I've always figured that for spinning rust, what kills them is power cycles. So, unless you're absolutely sure they won't be woken up, disable "sleep" mode. If you're sure nothing's going to wake them up, might as well turn them off completely, and the server also, saving a buck or two.
I don't think the performance and drive-lifetime hit of running rsync every 10 minutes would be good.
Zfs should have an edge in terms of atomicity as well, but in practice I'm not sure how much that matters. I _think_ it does matter but isn't perfect (zfs can't trick applications into doing atomic writes if they're not already, but it won't have _another_ worse layer of breaking atomicity like rsync must).