Treat every axis as being completely independent of the others and that will make you life so much easier. It means that you only need worry about air resistance, and in the case of the vertical axis, gravity.
To get a realistic movement due to air resistance, you'll need to use the
drag equation:
F_{d} = -0.5 \rho v^{2}AC_{d}
And also this formula:
F = ma
And this one:
v = u + at
Now, this part isn't mathematically correct, but if you consider only small steps forward in time, (1/1000 of a second for example) you can consider the drag force as constant over that time period, which allows you to do something like this:
ma = -0.5 \rho v^{2}AC_{d}
a = (-0.5 \rho v^{2}AC_{d})/m
a = -(\rho v^{2}AC_{d})/2m
This equation is getting to be a bit much, so to make it simpler, we can ditch lots of terms that don't change and replace them with a single constant. Specifically the frontal area of the ball (does actually change, but not very much), the coefficient of drag, the density of the air, and the mass of the ball. I'll use m = 0.06 Kg, diameter = 0.0635 m, air density = 1.2 Kg/m^3, coefficient of drag = 0.3 and t = 0.001 s. This leaves us with:
A = 0.03175^{2} * \pi = 0.003167
a = (1.2 * v^{2} * 0.003167 * 0.3 * 0.001)/(2*0.06)
a = - 0.0000095v^{2}
This will give the acceleration of the tennis ball in the opposite direction to it's direction of travel (if you consider the tennis balls direction to be positive). Or in other words, it's deceleration.
For this next bit, we need to come up with two equations, one for if the ball is traveling in the positive direction (ie: v>=0) one for traveling in the negative (v<0). First up is the positive:
v = u - 0.0000095u^{2}
And now the negative:
v = u + 0.0000095u^{2}
Notice I changed the v to u, because u is initial velocity and v is final velocity.
So there you have a simple way to calculate the new velocity after 1/1000 of a second. You simply plug in the old velocity as u and out pops the new velocity v. Oh, and the velocity is in meters per second.
Well those formulas can be used as-is for the horizontal plane, but what about vertically? There's also gravity to worry about. Well as luck would have it, this is way simpler to work out than the air resistance formula, because the acceleration due to gravity is always 9.8 m/s^2 down.
Just like before we need two different formulas due to the air resistance factor, firstly if the ball is going up (v>=0). In this one gravity and drag are working against the direction of travel:
a = -g - F_{d}/m
v = u - gt - F_{d}t/m
v = u - 9.8*0.001 - 0.0000095u^{2}
v = u - 0.0098 - 0.0000095u^{2}
And now if the ball is going down (v<0):
a = -g + F_{d}/m
v = u - gt + F_{d}t/m
v = u - 9.8*0.001 + 0.0000095u^{2}
v = u - 0.0098 + 0.0000095u^{2}
None of these equations take account of spin, which would alter the acceleration in mid air slightly, and especially so when the ball bounces. I'm afraid I wouldn't know where to begin to take account of spin.
