> That data size fits easily in memory, and without doubt on a single system. From that point you only need some degree of parallelism to maximize the performance. That being said when its 35TB of data, the answer starts to change.
Not at all, because data is being streamed. It could just as easily be 35TB and only use a few MB of RAM.
The IO bandwidth of the system will limit you more loading 35TB of data in ram on a single system, even if it is streamed. You'll need more than one disk, and network card to do this in a timely fashion.
Not at all, because data is being streamed. It could just as easily be 35TB and only use a few MB of RAM.