The terminology usually applies for cylindrical wave guides, where the cylinder can have any cross section, not only a circular one. One can prove from Maxwell's equations and the appropriate boundary conditions for the fields that each em. field can be expanded in a set of three types of eigenmodes that are
transverse electric (TE): The electric field has no component in direction of the cylinder axis,
transverse magnetic (TM): The magnetic field has no component in direction of the cylinder axis,
transverse electro-magnetic (TEM): both the electric and the magnetic field have no component in direction of the cylinder axis.
So, transverse is meant with respect to the cylinder axis. The TEM modes only exist if the cross section of the cylinder is not simply connected. A typical example is the usual caxial cable. These modes are particularly nice since they show no dispersion (in an ideally conducting wave guide), i.e., they admit an undistrubed signal propagation along the wave guide.