Hacker News new | past | comments | ask | show | jobs | submit login
Data-Intensive Text Processing with MapReduce (2010) [pdf] (umd.edu)
32 points by sytelus on Oct 18, 2015 | hide | past | favorite | 1 comment



The authors mention the problem of capacity increases in storage outpacing bandwidth growth.

This is mathematical inevitability at least for spinning disks, as long as capacity increases come from density increases. When density increases by a factor n, capacity increases by n^2, because the disk surface is two-dimensional. OTOH, the tracks under the head are linear, so the same density improvement of factor n only causes a corresponding bandwidth increase of factor n.

And seek times are governed by physics and haven't really improved much at all. Accelerating and decelerating a mass just takes a certain amount of time, and the faster you do it the more noise you generate.

A Winchester drive available for the Apple II had a capacity of 5MB and a transfer rate at the interface of 5MBit/second. With that it could transfer its contents in 8 seconds, not that the Apple II could accept it at that speed. Even assuming that the real disk transfer speed was 1/10th of that would still allow the whole contents of the drive to be transferred in a little over a minute.

A current 3 Terabyte drive with a generous 200MB/s interface takes 4 hours.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: