It has to do with the ability of the molecule to form hydrogen bonds or not. Water molecules are polar - they have an intrinsic dipole moment, whereas non-polar molecules don't. Water molecules will want to form hydrogen bonds with each other, and any other polar atoms they can, in order to minimize the total dipole moment of the system.
When you introduce a non-polar molecule to water, the surrounding water molecules form a sort of cage around the molecule, bonding with themselves, but in a much more restricted manner than if they could make bonds with the molecule. This has a large entropic penalty associated with it, and so the free energy, [itex]\Delta G = H - TS[/itex] is positive. Hence non-polar substances do not like to dissolve in polar solvents.