A first order phase transition involves a release (or intake) of latent heat. A second order transition does not.
There are no third or high order transitions. The name comes from a classification scheme by Ehrenfest that was in error. Though the terminology has stuck around, better names would be discontinuous transitions (which are "first order") and continuous transitions (which are "second order").
In a discontinuous transition the order parameter becomes zero discontinuously at the critical point, whereas in a continuous transition the order parameter changes to zero continuously up to the critical point, and then remains zero afterwards. An order parameter, in case you are unaware of the terminology, is a measurable quantity which is zero in one phase of the material (the 'disordered' phase) and is non-zero in the other phase (the 'ordered' phase).
An example of an order parameter is magnetization in a magnet. At H = 0, there is a continuous transition from a ferromagnetic state (a net fraction of spins are aligned) to a paramagnetic state (M = 0) at a transition temperature Tc as T is increased. At H nonzero, but T less than Tc, the sign of M is equal to the sign of H (but the magnitudes are unrelated). If you flip the sign of H, M will change discontinuously.
Similar considerations will apply to crystal phase transitions.