Tuesday, August 06, 2013

Spark and hadoop anyone ?


I should have published this a while back - slipped my mind :-)


Spark now supports batch mode on top of Hadoop v2 (Yarn) thanks to the work we did in Yahoo! Labs Bangalore !
The pull requests are here, here, here, here, here and here (to be applied in order). Have been committed to spark master and will be in 0.8 release !
Take it for a spin and try it out on PB of data !!

Sourangshu was involved in this along with me in getting this polished and submitted to spark.



1 Comments:

Anonymous Anonymous said...

Any work happening on the chess infrastructure offlate? My offer is still open.

- Daniel Shroff

3/13/2019 02:26:00 AM  

Post a Comment

<< Home