Building a rig for simulations

[Amok]

Hello!

As the title states, I want to get myself a new rig for simulating complex physical phenomena. Specifically, I'm going to run quantum chemistry/solid state physics/atomic physics calculations on it.
I'm planning to re-use some parts of my old PC, and buy some new ones.
I want to spend most of my money on the processor. The codes I use run in parallel, so I'd like to have at least 8 physical cores in my PC. I'm pretty sure I'm going to get an intel Xeon (maybe you can suggest something else). The question is, which one? There are tons of options out there!
Bottom line is, I want the most bang for my buck, I don't need to pay extra just to have that extra 4% more that the very best model gives me. I consider spending 1000$ to 1500$ on the cpu alone.

I guess another question is whether I should get several chipsets of few cores, or maybe just one with many cores (seems more reasonable since I plan on running OPENMP code on it, probably also more energy efficient). But then again, maybe you guys will lead me to a different conclusion.

I also posted this on a computer forum, but I think most people there build computer for gaming or graphical design, so input from scientists will be very welcome.

Any comments or suggestions are welcome!

Cheers!
:D

 

[phyzguy]

What about an array of Raspberry Pis? At $25 each, you could have 40-50 cores for the same money. I'm not sure if this is better approach, but I read that some research labs are considering this as an inexpensive way to get a large number of CPUs.

 

[pbuk]

The modern way to highly parallel computation is to use a Graphics Processing Unit (GPU). For $1,000 you should get 2,000 - 4,000 cores.

 

[Amok]

What about an array of Raspberry Pis? At $25 each, you could have 40-50 cores for the same money. I'm not sure if this is better approach, but I read that some research labs are considering this as an inexpensive way to get a large number of CPUs.

I actually thought of that, and it would be an interesting project, but I feel like I don't really have the necessary skills for something like that. Maybe in the future, when I have completed my PhD :D

The modern way to highly parallel computation is to use a Graphics Processing Unit (GPU). For $1,000 you should get 2,000 - 4,000 cores.

I know that, unfortunately most of the codes used in my field do not use GPUs yet (although I'm pretty sure it's going to happen in the near future). That's why I'm planning on getting a motherboard that can also support more than one GPU, so I cam upgrade in the futre.

 

[Routaran]

We had the same issue at work as well, in that the software we were using cannot work off the GPU. We settled on Dell T7900s at work to do the job but these things are not cheap, around $3000 per system ($5000 if you throw in a decent Tesla card)

Raspberry Pi cluster may work depending on the task but remember, a lot of these simulations require a stupid amount of RAM and/or generate huge scratch files. The Pi doesn't have a lot of RAM so you'll need plenty of disk space.

The USB3/Gigabit network can handle the throughput but budget flash drives today cannot really do more than 100Mbps, this will cost you time. A calculation that takes a couple weeks can potentially run into months. Just for fun, I once monitored all the traffic on the switch that our cluster is on and the number quickly ran into hundreds of gigs in a matter of hours.

It's why we often have RAID1+0 on these systems, so we can have sustained read/writes that are required and as little overhead as possible. You can to an extent deal with this by increasing the amount of RAM on each system (32/64/128G) but this is not really possible on a raspberry pi as far as I know.

You will need to find out exactly where the bottlenecks are on the software you are using, contact the manufacturer and ask them, they will be able to tell you. RAM, Processing capacity, hard drive read/writes, amount data sent over the network, etc. Then you have the base requirements for your computer system/cluster, and you can attempt to design something that will do the task and do it well.

You don't HAVE to dish out a crap tonne of money but for this type of work, unfortunately you sometimes have to.

A cluster of Raspberry Pi costing a thousand bucks is just insane for bitcoin mining or hacking your neighbour's wifi password because those tasks just need lots of cpu power, they need very little ram and very little hard drive access.
The raw processing power of 50-100 raspberry pi devices is impressive. But specialized tasks like scientific computing, in my opinion, aren't very good on these devices.

 

[Nidum]

http://www.thestar.com/news/insight...rk_gives_researchers_supercomputer_power.html

 

[DrZoidberg]

How many hours a day do you plan to run those calculations? If the rig would be running only a fraction of the time it may be cheaper to rent processing power or share a rig with other researchers.

 

[bigfooted]

I recently got a new HP z840 at work. It has 32 cores (Intel Xeon E5) and 256Gb of memory. It was a factor 10 more expensive than your budget, but maybe you can get a second hand Z800 or similar from a company where you can buy rented office hardware. I have an old 12-core Z800 and it is still considered quite fast.
Some thoughts:
If you run full-time computations on such beasts, the electricity bill is not negligible anymore.
If you run computations on a single machine, parallel programming in openmp is a negligible effort compared to parallel programming for multiple machines.
If you run on multiple machines, the connection between the machines becomes a bottleneck, and you need to invest a lot of money to fix this. If you are not building a professional cluster, I would go for a single machine.
If you are on a university, check their resources. Buy a nice laptop and log into their beasts from the comfort of your own home.