Hacker News new | past | comments | ask | show | jobs | submit login

He should seed a torrent file



Torrents aren't great for random access which this app is doing.


Bittorrent can do random access just fine, especially with v2 which enables content verification in 16KB chunks. Clients tend to be throughput oriented which leads to sub-optimal latency, but that's not a limitation of the protocol.


I guess just fine is a matter of opinion. Connection setup and piece bitmap tracking seems like more overhead than HTTP range requests.


Connection establishment in bittorrent might be marginally higher overhead than HTTP, but not by much and the cost is highly amortized if you're doing many range requests in the same torrent. There is indeed more overhead from HAVE messages and such, but that's just because bittorrent is doing a lot more than HTTP to enable P2P. It's still not that bad as a percentage of the payload traffic.


But at least we could get a copy without incurring a massive s3 bill.


Can we really? For torrent to work you still need someone to host it in full at all times. Who will host a 40GB file that will be out of date in a week? The website provider will, so we’re back to square one.


No, you don't. You just need N people to host 1/N of the file some of the time. The only person who needs to upload it in full is the original creator.


I would like a copy of that sqlite database to play with.

Having that all preprocessed would save a bunch of work compared to getting it from Wikipedia itself


IPFS?


Second that. I have no hands down experience with it, but I've seen seemingly more complex apps deployed on IPFS https://docs.ipfs.io/concepts/usage-ideas-examples/




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: