Hacker News new | past | comments | ask | show | jobs | submit login

I thought storm worked on infinite streams? Does this support that?



It should. It depends on which reducing function you choose -- the map step doesn't consume the stream, it just set up the mapped function to be called as you reduce. If your reducing function is lazy, then it works on infinite streams.


This works on collections, which are not infinite. If I read correctly, sequences (a higher level abstraction) fall back to their curent implementation.


I believe the existing pmap and preduce functions work in parallel over lazy streams by chunking work and parcelling it out. Depending on your use case, this is not necessarily ideal.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: