The impulse delivered in a punch has very little to do with the speed or acceleration of the fist, and much more to do with what the puncher does with the rest of his/her body. A trained person can do much more damage with an elbow strike than with a punch, yet the speed and acceleration of the elbow is far less than he can achieve with his fist. Why is that?
You can think of this in terms of impulse, energy, or momentum. Any way you look at it, you have to apply a force for a period of time or over a distance of travel. Any force imparted has to be transmitted to an equal and opposite force reacted by the ground. Nearly all the training to deliver an effective strike is aimed at maximizing the reaction force transmitted to the ground.
To get to the ground, the force has to follow an efficient load path, or it will be absorbed by your body, which is the connection between the force delivered to the victim and the reaction to the ground. So you learn how to shape that load path to keep it as short and as simple as possible, and how to make it pass through the strongest parts of your body.
The other important thing is delivering an effective strike is to maximize duration of force. That is why in Jujitsu I was always train to never deliver a strike to a person’s face. Aim for the back of the head. The face is just in the way. Punch through the face and head.
Nothing can demonstrate this better than the “one inch punch” made famous by Bruce Lee. It is possible to deliver nearly full punching power with only one inch of fist acceleration distance if you do everything correctly in the rest of your body. This was never meant to be a credible fighting technique, but rather a training method to remove the speed and acceleration of the fist from the equation all together and focusing on training the rest of the body to effectively develop the reaction force to the ground.
So if you want to understand the physics of delivering an effective punch, learn how to draw a free body diagram and understand the load paths through your body. Speed and acceleration are important, but more so for what the rest of your body is doing than for your fist.