Should I start using a parallel foreach everywhere possible?

[SlurrerOfSpeech]

I recently learned about C#.NET's Parallel.ForEach function and it made me think that this is the new hammer I have that will make everything look like a nail. Under the covers, the parallel foreach figures out how to use the optimal number of threads (based on your machines CPUs and cores) to accomplish the tasks in the loop in the least amount of time possible. Of course, it only makes sense to use this if

• the tasks are independent
• there is no specific order in which the tasks need to be completed

But that's easily the case for 75%+ of the instances of when I need to write a loop.

Am I making any incorrect assumptions?

 

[FactChecker]

I would only use it for loops that take a significant amount of time (in addition to the two rules you mentioned.) There is additional overhead in splitting the loop up for separate processors. If there is very little to do inside the loop, a parallel foreach can take up to 10 times as long. There is also the concern that a future software mod will create some dependencies that make you change those parallel loops. So I would only use a parallel foreach if it will solve a processing time problem in the loop.

 

[jedishrfu]

The short answer is no. You should profile your program to see where the bottkenecks are consider using it to speed up blocks only.

Parallel programming done poorly can introduce so many hard to find bugs that you will rue the day when you have to fix them. Usually you get race conditions where some variable is used before it has the proper value. Basically your code needs tobe synchronized with itself to eliminate the race conditions using mutexes to control access to data.

I would recommend that you explore multi threaded programming to understand the risks involved.

 

[rbelli1]

Beware: Your first multi-threaded program will not work or will only work on the computer you developed it on at the time of day you developed it (phase of moon and what you ate for breakfast may be important too). This is normal. Just determine the reason and try again.

BoB

 

[newjerseyrunner]

No, you should only create threads in areas that take a while. Creating a thread takes about a quarter millisecond on a modern computer, a for each loop through a string takes way less than that. If the loop takes less than 100ms, don't bother threading.

Also be sure you understand: mutexes, volatile, and thread pooling.

Also be aware of what libraries are threadsafe and which aren't. For example, on Visual C++, std::string is NOT threadsafe by default, you have to link to a special version because it uses reference counting.

 

[rcgldr]

The other example

http://msdn.microsoft.com/en-us/library/dd460703(v=vs.110).aspx

is a better example, as file reads from even an SSD can't be done in parallel. The concept is that C# will determine the optimal number of threads depending on the code and system that the code is running on. For a simple summation, it probably won't help much, but the example is just that. If the math was more complex and cpu bound then multiple threads would help. You could always try a benchmark to see if it helps.

Locality of data within each core's local cache can also help in what would otherwise seem like a memory bandwidth limited process. I wrote a single and multi threaded (4 threads) version of bottom up (iterative) merge sort, and for 16 million 32 bit integers, the 4 thread version runs about 3 times as fast as the single thread version (0.5 seconds versus 1.5 seconds with Intel 3770k 3.5ghz cpu).

This isn't a new concept. Fortran compilers have included vectorization and multi-threaded extensions for decades.