Wrong. There is no "energy holding the nucleus together". Binding energy is exactly the amount energy nucleus LOST when it was formed from constituent protons and neutrons. This energy is simply MISSING. It is not there.
When fusion happens, the resulting energy release you see is exactly this binding energy being released.