The point in swinging is to increase the initial velocity of the object at the moment of release. That velocity is equal to the instantenous tangential velocity in circular motion. Swinging around allows the thrower more time to accelerate the object.
Initial velocity, of course, translates into increased distance travelled, as per the equations of motion.
If you'd rather think of it in terms of energy, then swinging around means that there is virtually an unlimited path(a closed circle) on which a relatively low force supplied by the thrower can perform work on the system(thrower+object) to increase its rotational energy.
At the moment of release, the rotational energy is transferred to the two components of the system(thrower+object) as kinetic energy. Higher kinetic energy means higher initial velocity, which, again, means farther throw.
Compare with throwing just by using one's arm's length - it would require much higher force to impart the object with the same amount of energy, as the distance over which the work can be performed is very limited.