Hacker News new | past | comments | ask | show | jobs | submit login

If writing performance is critical, why bother with deduplication at writing time? Do deduplication afterwards, concurrently and with lower priority?



Keep in mind ZFS was created at a time when disks were glacial in comparison to CPUs. And, the fastest write is the one you don't perform, so you can afford some CPU time to check for duplicate blocks.

That said, NVMe has changed that balance a lot, and you can afford a lot less before you're bottlenecking the drives.


Because to make this work without a lot of copying, you would need to mutate things that ZFS absolutely does not want to make mutable.


If the block to be written is already being stored then you will match the hash and the block won't have to be written. This can save a lot of write IO in real world use.


Kinda like log structured merge tree?




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: