Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
londons_explore
on Jan 4, 2024
|
parent
|
context
|
favorite
| on:
The One Billion Row Challenge
This assumes the file is already in page cache, and therefore already in RAM. The 0.3 seconds is the time to get it from RAM to the CPU L3 cache.
mgaunard
on Jan 4, 2024
|
next
[–]
That sounds like cheating. The exercise says to process a large file that's on disk. If the data is in RAM then that's a whole different game.
rockwotj
on Jan 4, 2024
|
parent
|
next
[–]
The benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW
mgaunard
on Jan 4, 2024
|
root
|
parent
|
next
[–]
That sounds like a benchmarking flaw.
But then, writing good benchmarks is an art in itself.
PaulDavisThe1st
on Jan 4, 2024
|
root
|
parent
|
prev
|
next
[–]
If the cache is not flushed, it will still be in memory between runs whether you mmap it or not.
marginalia_nu
on Jan 4, 2024
|
root
|
parent
|
next
[–]
That depends a lot on the OS. I would not count on a 13 GB file being cached in RAM after having been read through once.
brrrrrm
on Jan 4, 2024
|
parent
|
prev
|
next
[–]
Isn’t it run multiple times and the slowest is dropped? Why would you expect the file to be evicted from ram so quickly?
menaerus
on Jan 4, 2024
|
prev
[–]
That makes sense now, thanks.
Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: