What is the Most Efficient Way to Implement MPI in a Large Vector Algorithm?

[maverick_starstrider]

Hi,

I'm writing an algorithm in MPI which basically splits up a massive vector amongst different nodes (i.e. node 1 has entries 0-1,000,000, node 2 has 1,000,000 - 2,000,000, etc.) and basically my algorithm requires that some calculation be done on each element of the array and the result gets added to another component of a duplicate array of the same size and split up in the same way which, most likely, won't be located on the same node (i.e. after computing on element 1000 on node 1 I find that I have to add the result to element 1,000,102 which is on node 2). So basically, if each node has n components of the vector on it it's going to have to send the result to n different places AND receive lots of potential changes to its own section of the vector. Now my question is, what is the BEST way to do this? I'm torn between:

-after computing each element ISend the result (assuming it needs to be sent to another server) and then do a sweep of IReceives to see if there are any results from other servers that are trying to be sent to me.
-the same thing using buffered send (IBSend). However, memory usage is a huge issue (I'm basically going to make my vector as big as possible) so I don't know how many buffers I need. Should I set up one local buffer for each server (i.e. buffer 2 on server 1 is big enough for a single message and is reserved for IBSend's to server 2). But then what if I need to send to another server but there's still an outgoing message waiting to be received on the other side. I'll have to block until it gets picked up
-The final option I'm wondering about is MPI-2's windowing features and using the put command to simply place the result where it needs to be placed.

Can someone who knows a fair amount about MPI performance considerations help me determine which implementation will be the most effective. Any help is greatly appreciated

 

[lofgran]

.

Hi there,

As a fellow scientist working with MPI, I can understand your dilemma. It's important to consider performance when designing an algorithm for distributed computing. In your case, it seems like a combination of both ISend and IReceive and MPI-2's windowing features could be the most effective approach.

With ISend and IReceive, you can achieve better performance by overlapping communication and computation. This means that while one node is computing, another node can be receiving data, reducing the overall execution time. However, as you mentioned, this approach requires careful management of buffers and can lead to blocking if not done properly.

On the other hand, MPI-2's windowing features can provide a more efficient and flexible solution. By using MPI_Put, you can directly place the result where it needs to be without the need for explicit communication. This can save time and resources, especially if you have a large vector and need to constantly send and receive data between nodes.

Ultimately, the best approach will depend on the specific requirements and constraints of your algorithm. I would suggest experimenting with both approaches and measuring their performance to determine which one works best for your particular case. Also, don't be afraid to consult with other experts in the field or seek out resources and tutorials on MPI performance optimization. Good luck with your algorithm!