Hacker News new | past | comments | ask | show | jobs | submit login

Its not that odd, spark is very similar to the hadoop way of doing things and there are a few other projects like it that are attempting to be a better hadoop. So I dont think that much was lost by not mentioning it.



How is Spark similar to the Hadoop way of doing things ?

It operates very differently from Hadoop for us. SparkSQL allows third parties apps e.g. analytics to use JDBC/ODBC rather than going HDFS. And the in memory model and ease of caching data from HDFS allows for different use cases. We do most work now via SQL.

Combining Spark with Storm, ElasticSearch etc also permits a true real time ingestion and searching architecture.


Spark is a more general data processing framework than Hadoop. It can do map-reduce, can run on top of Hadoop clusters, and can use Hadoop data. It can also do streaming, interactive queries, machine learning, and graph processing.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: