basically the interesting properties of QCD exist because the gluons interact with each other. one of these properties is that the strong force becomes stronger with increasing distance, or equivalently, with decreasing energy. in a nucleus you have relatively low energies/large distances, so the strong force is very strong. this means when you try to calculate something, your equations diverge to infinity.
to explain this in more detail, first consider QED.
in QED, 'bare charges' (e.g. an electron) are screened by virtual processes - emission of virtual photons, e+e- loops - effectively reducing the electromagnetic coupling constant. if you increase energy of whatever measuring system you have, equivalently decreasing distance, you are seeing less of this screening and so the coupling constant is larger. This is why the electromagnetic coupling increases with energy.
in QCD, you have an SU(3) gauge symmetry which gives rise to properties different to the U(1) symmetry of QED. importantly here, it means that the gluons interact with each other (photons do not interact with each other). as a result, the virtual screening in QCD has the opposite effect to QED; with increasing energy/decreasing distance, you are seeing more screening of the 'bare' color charge.
The result is that the strong force gets stronger with increasing distance. So at low energies/large distances, you actually have a very strong QCD coupling. These are the conditions that exist in atomic nuclei among other things.
now since any process in a QFT can be written as an expansion in terms of these coupling constants, and there are infinite terms, if the coupling is ~1 (as it is for QCD at low energy) then the expansion diverges and the process cannot be calculated in the normal way. With QED, or with QCD at high energy, this is not a problem since is the coupling is << 1 and so the terms in the expansion become smaller and smaller can be neglected.
