In general it has to do with the irreversibility of the reaction. In primary cells if you put the energy back into the system, it will not go back to the original state. Say, in Leclanche battery, instead of reversing the reaction you will just decompose water present (and produced in the battery reaction) into hydrogen and oxygen - leaving ZnCl2 and Mn2O3 intact, so the battery doesn't recharge.
Thermodynamically - when you try to recharge primary cell it won't get to the original state, but to some local energy minimum. Secondary cells are those that don't have such competing local minimum, so they can be pushed only back to the charged state. Note, that asking "why secondary cells behave this way" is putting the cart before the horse - we call cells that behave this way "secondary", so it is behavior that classifies the cells, not the the classification that tells us how the batteries behave.