The Earth is gravitationally bounded to the sun. If the sun were to suddenly disappear (say, it got hit by an anti-sun, and all the mass annihilated completely), the Earth wouldn't be bounded anymore, and it would fly off into space. Would the speed of the Earth now be more than the tangential velocity it had when it was in orbit around the sun? Or where would the (previous) potential energy go? A similar thought experiment could be... Take a spring, and attach a mass to each end. Now stretch the string, by moving the masses apart. The system has gained some potential energy. If the spring were to suddenly disappear, the masses would have no way to reclaim the potential energy they had when the spring was there. Where does the (previous) potential energy go? Is the radiation from the spring (if again, it disappeared through annihilation) more energetic? In this example, I can easily imagine the to masses to be at rest after the spring is gone; hence the potential energy does not convert to kinetic energy here.