Can anyone access supercompute time?

[aheight]

TL;DR Summary

What does it take for someone not affiliated with academia to get time on a supercomputer?

I'm working on a problem which can be CPU-intensive and was wondering what it would take for me to get time on a supercomputer to run it and if so, how would the speed of a typical supercomputer compare to my 4.5 GHz machine? It's a relatively simple Mathematica program to find polynomial roots.

Thanks for reading guys.

 

[phyzguy]

Basically, if you don't have access to a supercomputer through some project, you would have to pay to access a large machine. Most supercomputers are just large numbers of computers similar to the computer you own. So unless you can parallelize your problem so it can run on many cores at the same time, it will not run any faster on a supercomputer. Do you know how to parallelize your problem?

 

[aheight]

Basically, if you don't have access to a supercomputer through some project, you would have to pay to access a large machine. Most supercomputers are just large numbers of computers similar to the computer you own. So unless you can parallelize your problem so it can run on many cores at the same time, it will not run any faster on a supercomputer. Do you know how to parallelize your problem?

Thanks for that. It's surprising to me. I thought supercomputers run a lot faster. I can easily parallelize it via ParallelTable in Mathematica for multiple runs but I can't parallelize it for a single run of Newton Iteration of a single root. .

 

[Vanadium 50]

The good news is that anyone (pretty much) can submit a proposal to run on various government-owned supercomputers. The bad news is that your proposal will not be successful.

1. The proposal needs to advance the mission of the agency owning the supercomputer. DOE's NERSC? Needs to be related to Department of Energy research. OLCF? Either that or needs to advance supercomputing somehow.
2. The proposal process is heavily oversubscribed. Perhaps 3x as much time as requested as is able to be awarded.
3. Your program has to be very parallel. My record is over 3 million simultaneous processes.
4. The architectures for the largest machines don't run Mathematica. Furthermore, because desire to use these machines is so intense, an enormous premium is placed on efficiency. That usually means running code you control, not code Wolfram controls.
5. Time is given out in chunks of millions of CPU hours. Your project sounds too small.

However, you do have an option: commercial clouds. Mathematica can use Amazon's EC2. Cost varies by time of use, the amount of memory you need, the number of CPU cores, etc. You will end up in the ballpark of tens of cents per hour used: i.e. if you had ten computers working for a day, expect a bill of $20-100.

 

[aheight]

However, you do have an option: commercial clouds. Mathematica can use Amazon's EC2. Cost varies by time of use, the amount of memory you need, the number of CPU cores, etc. You will end up in the ballpark of tens of cents per hour used: i.e. if you had ten computers working for a day, expect a bill of $20-100.

Thanks. I googled it. Would you happen to know if the performance would be much better than my machine? For example, if a (non-parallelized) Newton Iteration on my 4.5 GHz machine took 450 sec, how much faster would it run on the EC2?

 

[Vanadium 50]

Can I benchmark code I don't have and even if I had it, I would have to pay to benchmark it? Um...no.

EC2 has a dozen different types of processors and of order two-dozen variations on memory and effective CPU count. There is no single number. There's real work involved, and frankly, you need to be the person doing it.

The CPU cores are 8175M's. You should be able to figure out the relative performance. However, as the clock is slower, it will likely emerge that it takes somewhat longer.

But let's step back and look at the big picture. You have a job that takes ten minutes to run. Running it on a supercomputer where it sits in the queue for a week and then runs in one minute - how does this help you? Even if it sits in the queue for only an hour. How does this help?

 

[aheight]

But let's step back and look at the big picture. You have a job that takes ten minutes to run. Running it on a supercomputer where it sits in the queue for a week and the runs in one minute - how does this help you? Even if it sits in the queue for only an hour. How does this help?

Ok I get it now. Was just considering alternatives. My work is not mission-critical. Thanks for helping!