BigMacnFries said:
I have read numerous books that describe the formation of a solar system from a cloud of gas. They state that as the gas contracts it rotates and as it rotates it flattens out into a disk. I do not understand why the gas molecules start to rotate
I think it's fair to say that astronomers aren't sure either, but it probably has something to do with tidal forces (i.e. forces that pull differently on different parts of the cloud) from the galaxy and other nearby objects. Remember that it only takes a tiny amount of rotation when the cloud is large to make it rotate quickly when it's small -- angular momentum is conserved.
how the gas molecules decide which way they are all going to spin
Randomly is the short answer. The long answer would involve correlations with angular momentum on larger scales. At the largest scales, however, the sources of angular momentum in collapsed structures are almost certainly random. If this weren't the case, the universe would have a net angular momentum and the cosmological principle would be violated.
and once the cloud is spinning why it flattens out.
This is a tough question to answer from first principles. Basically, it has to do with the fact that a disk is the lowest-energy configuration for a strongly rotating system. If there were no net angular momentum, the lowest-energy configuration would be a sphere. The latter case just follows from symmetry. In the absence of a preferred direction or axis, there's no reason the density should be higher in any direction or along any axis. Thus, a sphere. If there is net angular momentum, however, there is a preferred axis (i.e. that around which its rotating), so one would expect the system to form a shape that had symmetry about that axis. This could be a disk or, perhaps, a spheroid, but it just turns out to be a disk.
In real systems, however, the single pseudovector (angular momentum) is not enough to describe the motions of the system. One should also consider tensor terms and complicated gravitational interactions in order derive the final state. You probably need not worry about this, however.