So a phase transition occurs when a system may minimize its free energy by spontanously breaking some symmetry. At least that is what I have learnt from my current course so far. However I also did another course concerning phase transitions. In this case phase transitions were always explained by some kind of instability, e.g. the divergence of for instance the electric susceptibility function χ. How does all this go together with the interpretation of spontanous symmetry breaking as caused by minimizing the free energy?
The free energy pictues of first and second order transitions are given in http://www.cmp.caltech.edu/~mcc/BNU/Notes2_2.pdf . A first order transition is something like the liquid-gas boundary. A second order transition is like the critical point where first order boundary ends, and there is no distinction between liquid and gas.
In my opinion, the appearance of long range correlations is really at the heart of phase transitions while the precise sense in which a symmetry is broken depends on the representation chosen.
Spontaneous symmetry breaking and divergence of a (suitably defined) susceptibility are two aspects of the same thing. You can think of the susceptibility as the coupling between an external field and an "order parameter". When the susceptibility diverges the zero-field expectation value of the OP is non-zero and symmetry is spontaneously (without external field) broken. In all cases the system will try to minimize free energy. Above the transition in a field the OP will be non-zero (due to a finite susceptibility) to minimize the free energy. Or you might interpret the susceptibility as the parameter that describe the optimum response (in terms of free energy) of the system to an external field. This is evident when you look at how you can shift the phase transition temperature up and down by applying external fields that stabilize (decrease free energy) or destabilize (increase free energy) the OP. Landau theory of phase transitions is great for this.
I just wanted to point out that this is a tricky point although the picture is not wrong. You can have superpositions of states were the order parameter has different directions but equal magnitude, so that the total expectation value of the OP is zero. This is especially relevant in superconductors where the OP isn't an observable and states with definite value of the OP correspond to unphysical states with fluctuating particle number.
Thanks for pointing that out. I guess you are talking about d-wave superconductivity. For standard BCS one might use the density of Cooper pairs as OP. That should be a scalar without direction. In any case, it is not quite clear to me what the "susceptibility" for superconductivity could be. SC can be suppressed by external magnetic fields, but I do not know of any external stimulus that would enhance it. Hydrostatic pressure works indirectly, I suppose, via the electron density and Fermi level. So does chemical doping. This simple picture works quite well for magnetic phase transitions.
Well, what I had in mind in superconductivity is not actually direction but the phase of the condensate. The "susceptibility" which diverges in the normal to superconductor transition is discussed e.g. in R. Mattuck, A Guide to Feynman Diagrams in the Many-Body Problem, Chapter 15.4.