We have a proprietary algorithm for assigning foods a "suitability score" based on a user's personal health conditions and body data.
It used to be a fairly slow algorithm, so we ran it in a hadoop cluster and it cached the scores for every user vs. every food in a massive table on a distributed database.
Another developer, who is quite clever, rewrote our algorithm in C, and compiled it as a database function, which was about 100x faster. He also did some algebra work and found a way to change our calculations, yielding a measly 4-5x improvement.
It was so, so, so much faster that in one swoop we eliminated our entire Hadoop cluster, and the massive scores table, and were actually able sort your food search results by score, calculating scores on the fly.
It used to be a fairly slow algorithm, so we ran it in a hadoop cluster and it cached the scores for every user vs. every food in a massive table on a distributed database.
Another developer, who is quite clever, rewrote our algorithm in C, and compiled it as a database function, which was about 100x faster. He also did some algebra work and found a way to change our calculations, yielding a measly 4-5x improvement.
It was so, so, so much faster that in one swoop we eliminated our entire Hadoop cluster, and the massive scores table, and were actually able sort your food search results by score, calculating scores on the fly.