The entropy density s of a radiation field of temperature T is s ~ T3. The entropy S in a given comoving volume V is S = s V. Since the comoving volume V increases as the universe expands, we have V ~ R3. And since the temperature of the microwave background goes down as the universe expands: T ~ 1/R, we have the result that the entropy of a given comoving volume of space S ~ R-3 * R3 = constant. Thus the expansion of the universe by itself is not responsible for any entropy increase. There is no heat exchange between different parts of the universe. The expansion is adiabatic and isentropic: dSexpansion = 0.