The discussion centers on the behavior of MPI collective operations, specifically the reduce operation. It clarifies that nodes do not always send data directly to the root; instead, they can utilize intermediate nodes for data reduction. The use of MPI_ALLREDUCE is emphasized as a method to achieve this in two steps, allowing all group members to receive results simultaneously. The MPI Forum documentation is referenced for further details on the operations of MPI_ALLREDUCE and MPI_REDUCE.
PREREQUISITESResearchers, developers, and engineers working with parallel computing and MPI who need to optimize collective operations and data reduction strategies.
ektrules said:Or is there some structure where a node will receive data from a few other nodes, perform a reduction, then pass the intermediate results to other nodes?
If comm is an intracommunicator, MPI_ALLREDUCE behaves the same as MPI_REDUCE except that the result appears in the receive buffer of all the group members.