I think it's also worth considering the relative mass of the bullet to the gun when we treat the explosion initiating the gunshot as F. Using Newton's third law, we know that the force applied to the bullet by this explosion will be equal and opposite to that applied to the gun. This means that the resulting acceleration will depend on each object's mass.
Let's call F 100N, m of gun 1.18Kg, and m of bullet .0075Kg. Now using the formula: F=ma, we can determine that under these circumstances, the bullet will accelerate at 13333.3m/s^2, and the gun will accelerate at 84.75m/s^2. Although these values may not be exactly true to real life, the relationship they demonstrate is accurate and significant. The masses used reflect that of a generally realistic relationship using data found online (didn't bother with force which was constant in comparison), and taking that into consideration, the answer can be seen. If we choose to measure an object's relative ability to cause damage in the amount of kinetic energy each element carries, we can work out the following:
Ratio of mass of gun to mass of bullet: 157.333:1
Ratio of acceleration (therefore final velocity given constant distance) of gun to bullet: 157.325:1
K=.5mv^2
In the formula, since v in terms of K has a higher influence than m given its degree, the amount of kinetic energy the object in question possesses will depend most significantly on its velocity. For this reason, a bullet traveling at a higher velocity than a gun even with as little mass it has in comparison to the gun, will have more kinetic energy, and will generally cause more damage. The values found indicate this relationship.