First recall the radioactive decay is a process by which an unstable nucleus emits particles such that the sum of the new nucleus and emitted particle are more stable than the original particle.
The helium nucleus is doubly magic, and it is very stable. Thus for large nuclei that are unstable because they are too big, its a ideal particle to emit.
In chemistry you learned that when an atom has a certain number of electrons (2,10, 18, ...) it is extremely stable. This is why the noble gases don't react chemically.
A similar thing happens in the nucleus of an atom. The interaction is more complex because you have to forces at play (instead of one), and you have both protons and neutrons (instead of just electrons). It turns out that when an atomic nuclei has a certain number (2,8,28,50 ... ) of protons or neutrons will tend to be more stable. Nuclei where both the protons and neutrons are on of these "magic" numbers are extremely stable. Helium for is one of these isotopes. Some other examples are oxygens-16 and lead-208.