Start off with a circular orbit. If an outwards perpendicular force is applied to the object, it's path becomes non-circular, and once the path is non circular (and outwards) a component of gravity will slow the object down. If the force remains perpendicular to the object's motion, then no work is done, so this only changes the shape of the orbit. The distance can no longer be increased once the objects speed is reduced to zero, which happens when the objects gravitational potential energy equals it's total energy.
Take the case where the perpendicular force is continuously adjusted so that the object travels in a straight line that doesn't intersect the earth. The force is perpendicular to the objects motion, so the force does not affect the objects speed. However as the object increases distance from the earth, the direction of gravity opposes the objects motion, but with decreasing strength (G M m / r^2). Assuming the object is not initially traveling at or above escape velocity, it slows down and eventually stops, and then begins a return path. The perpendicular force could be adjusted to continue to maintain a straight line, while the object moves back and forth along the straight line.
If the outwards force is radial, then once the object's path is no longer circular, the outwards radial force opposes gravity, so the object slows down less or if the force is great enough, the object speeds up. If you think of this as a single impulse to produce some amount of radial velocity, then the final velocity = sqrt(original velocity^2 + radial velocity^2). If the the force was in the direction of the object, than the final velocity = original velocity + change in velocity, which is more efficient. Say you wanted to double the velocity, with the radial thrust, the radial velocity would need to be sqrt(3) ~= 1.73 x original velocity, while with thrust in the direction of motion, the change in velocity would need to be 1.00 x original velocity.