<blockquote style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;" class="gmail_quote">since mapReduce assumes that all the code (and the data) is in place in the respective nodes.<br>
</blockquote><br><br>As far as I can tell, the Hadoop, the open source implementation of map reduce, doesn't require your map reduce code to be in all nodes. It copies the jar files of the your application to the nodes that would execute the code. <br>
<a href="http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Job+Submission+and+Monitoring">http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Job+Submission+and+Monitoring</a><br><br>Data is also generally available via a network protocol such as http. <br>
<br>I am wondering if distribution tools (such as Cabal) would be preferable to hadoop model. Instead of having the framework copy the code why not use the existing tools to install the code prior to running the map-reduce? <br>
<br>daryoush<br><br><div class="gmail_quote">2009/2/25 Alberto G. Corona <span dir="ltr"><<a href="mailto:agocorona@gmail.com">agocorona@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Galchin,<br><br>Maybe you are asking not only about remote execution, but also mobility of code. This is a problem that is previous to mapReduce, since mapReduce assumes that all the code (and the data) is in place in the respective nodes. In fact, the distribution of resources in order to efficiently use mapReduce is a design problem that the google people has done by hand.<div>
<br></div><div>But my intuition says that there are a general algorithm for distribution of code, data, bandwidth and resources in general that moves around at execution time to achieve better and better performance for a given grid of nodes and for any task, for example, a mapReduce task. I would be very interesting to read something about this. </div>
<div><br><div>I know that some efforts have been carried out the past , for example mobile haskell <a href="http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf%C2%A0" target="_blank"></a></div>
<div><br></div>
<div><a href="http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf%C2%A0" target="_blank">http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf </a> </div>
<div><br>which is a first step for this goal but I this has been discontinued and the source code is not available.<br>
<br>2009/2/25 Galchin, Vasili <<a href="mailto:vigalchin@gmail.com" target="_blank">vigalchin@gmail.com</a>><div class="Ih2E3d"><br>><br>> Hello,<br>><br>> Here is an interesting paper of Google's MapReduce reverse engineered into Haskell. I apologize if already posted ..... <a href="http://www.cs.vu.nl/%7Eralf/MapReduce/" target="_blank">http://www.cs.vu.nl/~ralf/MapReduce/</a><br>
><br>> Kind regards, Vasili<br>><br></div><div class="Ih2E3d">> _______________________________________________<br>> Haskell-Cafe mailing list<br>> <a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a><br>
> <a href="http://www.haskell.org/mailman/listinfo/haskell-cafe" target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br>
><br><br></div></div></div>
<br>_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/haskell-cafe" target="_blank">http://www.haskell.org/mailman/listinfo/haskell-cafe</a><br>
<br></blockquote></div><br><br>