Hacker News new | past | comments | ask | show | jobs | submit login

Are you looking to pre-generate the results or generate them dynamically? I think MapReduce will only be effective for the former.



There are no map-reduce implementations that work in real-time? I thought Google search worked like that. I do want it to work dynamically.


MapReduce is used to build the search index (constantly), and your searches are run against the index.

You might find some of these links useful:

http://wiki.apache.org/hadoop/ProjectDescription

http://wiki.apache.org/hadoop/HadoopIsNot


You can use map-reduce on your search by partitioning up the starting keyspace alphabetically (26/num nodes), although with such an even partitioning the search will probably be unbalanced. But with some simple metrics on the starting letter frequency of words you could define a more balanced partitioning.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: