The speed of the bullet and the recoil of the gun are given by conservation of momentum
mV (Bullet ) = -Mv (gun)
If M >>>m then |V| >>>|v|
Kinetic energy is proportional to v2 so, as the gun gets more and more massive, less and less energy is transferred to the gun (approaching zero) and, eventually, it all goes into the bullet's K.E..
You only get equal energy if gun and bullet have the same mass - unlikely.
The work done on the wall by either striking against it will be equal to the K.E.. Actual damage will be more for the bullet (not surprising or anyone who fired a gun would be dead !) That answers your question, I think.
Further explanation involving the force is more complicated. Let's have a sandbag, rather than a concrete wall. Assume the sandbag absorbs all the energy (inelastic collision) in both cases. In both cases, there is the same momentum change (∆P) involved, coming to rest.
∆P = force X time (the impulse, which is equal in both cases)
The work done will be
W = force X distance moved (which is very different for each case).
There are factors like the contact area, which usually will reduce the pressure from the stock of the gun compared with the bullet but if the stock were a sharp rod - of the same area as the bullet, you could say that the penetration would be proportional to the K.E..
(KE = force X penetration distance).
But I think you would need to specify the modulus and density of the sand (or whatever) to analyse accurately any further but you can say that there is much more work carried out by the bullet than by gun penetrating the sand. The actual forces in each case can't be calculated without more information, I think.