It's easiest to explain this step by step.
In special relativity (no quantum mechanics yet), the "mass shell" or "mass hyperboloid" is the set of all energy-momentum combinations for a particle which give the correct rest mass for that particle, if you transform to the frame of reference in which the particle is at rest. If a particle has rest mass M, and a particular state of motion (energy-momentum combination) was equivalent to having some other rest mass M', then you would know that this isn't a physically possible state of motion for that type of particle.
A "mass-shell condition" is an equation which tells you whether a particular energy-momentum combination is on the mass shell or not, and thus whether it's physically possible.
In quantum mechanics, particle states are represented by wavefunctions, which encode energy and momentum of the particle in their frequency and wavelength. So the mass-shell condition is now expressed in terms of the properties of the wavefunction.
Also in quantum mechanics, one of the ways that you obtain the probabilities of events is by summing over all sorts of possible histories (with particles going this way and that, transforming, emitting and absorbing, etc), with each history making a contribution to the final probability. Mathematically, these "sums over histories" are integrals. Integrals are often easier to solve with a change of variables, or by changing the range of a variable that is summed over. You might be trying to integrate a function over the real numbers from -infinity to +infinity, but it will be easier to integrate it over all the complex numbers, and you have some theorem which tells you that the answer is the same - that sort of thing often happens.
When you are doing the quantum sum over histories, and you change the integral to an equivalent one that is easier to solve, that can mean that the particles appearing in the middle of the history take unphysical values for their properties. That's the result of doing the integral over a broader range of values. This can mean that the "virtual particles" "go off shell". But the particles that are observed, at the beginning and at the end, have to be "on shell", because that part of the calculation directly corresponds to reality.
(There are also ways to do the sum over histories where you stay physical and "on shell" from beginning to end.)
Finally, when we get to a quantum string theory like the heterotic superstring... the energy of the string is in its vibrations. And since it is a quantum string, again we have a wavefunction, so it's a little complicated. But again, you have mass-shell conditions, which tell you what sort of string wavefunctions correspond to physically meaningful quantum string states.
So to sum up, a mass-shell condition is a technical statement about what sort of quantum wavefunctions correspond to genuine physical states; and for the purposes of calculation, you will sometimes use wavefunctions that temporarily go off-shell, because that makes it possible to calculate the integrals, but probably it's just a mathematical technique and not a physical reality.
(I say "probably", because there is a lot of disagreement about the physical reality behind quantum mechanics, and it's logically possible that there is an "ontological interpretation" of quantum mechanics in which going off-shell actually happens.)