Parallel Processing 2

welatiger

i asked a question that " i need to make a parallel processing " but still wants to know from where i start



I need to learn

Parallel computing processes i.e. I hope to build cluster

Linux Parallel Processing Using Clusters we have 16 processors

mgb_phys

A good place to start is probably MPI - a library for linking programs runnign on clusters.
The wiki page is a bit techncial http://en.wikipedia.org/wiki/Message_Passing_Interface but has some examples, the home page might be useful http://www.mcs.anl.gov/research/projects/mpi/

maverick_starstrider

MPI is a library for writing code that runs on a parallel implementation. However, if you're trying to MAKE a parallel implementation (i.e. make your own cluster) that's not what you need. I'd start looking through linux websites on cluster building (it also depends a lot on the cluster you wanna build)

mgb_phys

Good point !

What we found was that at 16nodes pretty much anything will work - just buy whatever CPU is fastest/$ at the moment, get a MB with Gigabit ethernet and as much ram as you can afford. Look at duals when Dell are having a sale, 8*2cpu is often faster than 16*1cpu because half of your interconnects are super fast.

There is an O'Reilly book "Building Beowulf clusters" but it is out of date and wasn't very good when it was new.

For larger clusters (>64nodes) it's worth buying decent rack mount hardware from a proper vendor, otherwise you never have a system that is stable enough to complete a job before some fan fails and a machine hangs.
Racks, network and cooling start to cost you as much as the HW at this point.

Learn about network tuning and TCP packets, buy decent switches don't daisy chain home grade ones. If you need lower latency than ethernet it's probably time to pay the experts.

DavidSnider

If you are interested in the software development aspect of it I would recommend just buying CPU time on one of the many "cloud computing" networks. Look into Amazon EC2 or Sun's Grid.

Also, functional programming using Haskell, Erlang, Standard ML is ideal because of its "no side effects" nature.

maverick_starstrider

I feel like there's probably a sizable hobby community for this kind of stuff. If you can find the right website there's probably a wealth of information.

welatiger

thank you so much