"Reads need to be fast, even though they are rare. There are generally two approaches to dealing with this. The first is to write efficiently, so the data isn’t read-optimized per-series on disk, and deploy massive amounts of compute power in parallel for reads, scanning through all the data linearly. The second is to pay a penalty on writes, so the data is tightly packed by series and optimized for sequential reads of a series."
As the little girl says in the GIF: why don't we have both? Write to a write-optimized store of limited size that requires full access during reads, and re-write that into a read-optimized format hourly or daily. Because it's limited in size, you won't care that the most recent data isn't very efficient for reading, or isn't particularly compact.
As the little girl says in the GIF: why don't we have both? Write to a write-optimized store of limited size that requires full access during reads, and re-write that into a read-optimized format hourly or daily. Because it's limited in size, you won't care that the most recent data isn't very efficient for reading, or isn't particularly compact.