jazhemar said:
Thank you! But, how do I know for which of those four roots are the imaginary parts positive?
So let's examine the equation
$$z^2=\frac{-a^2b^2-1\pm\sqrt{a^4b^4-6a^2b^2+1}}{2b^2},$$
which describes all four poles of $f(z)$. First of all, note that the discriminant
\begin{align*}
a^4b^4-6a^2b^2+1&=a^4b^4-2a^2b^2+1-4a^2b^2 \\
&=(a^2b^2-1)^2-4a^2b^2 \\
&=(a^2b^2-1-2ab)(a^2b^2-1+2ab) \\
&=(a^2b^2-2ab+1-2)(a^2b^2+2ab+1-2) \\
&=((ab-1)^2-2)((ab+1)^2-2).
\end{align*}
This might help you get a handle on where the roots lie, given the locations of $a$ and $b$.
Now, I don't know what the RHS is fully, but since the quadratic equation in $z^2$ has all real coefficients, then both of these solutions, if they are complex, come in complex-conjugate pairs. As a matter of fact, if any of these roots are real and positive, then the original integral blows up along its contour. I don't think this happens, though, and here's why:
\begin{align*}
(a^2b^2+1)^2&=a^4b^4+2a^2b^2+1 \\
&>a^4b^4-6a^2b^2+1,
\end{align*}
and hence
$$a^2b^2+1>\sqrt{a^4b^4-6a^2b^2+1}.$$
So, if the discriminant is positive, the square root of its magnitude is less than the negative thing being added to it. So when you take the positive sign, you get a negative, and when you take the negative sign, you get something even more negative. So there cannot be any positive real values of $z^2$.
I suppose there could be negative real roots, in which case the contours I suggested in my previous post would run right over them; you can either change your contour to be only in the first quadrant (You can't go on the imaginary axis, because a negative value of $z^2$ would square root to be on the imaginary axis! You'd need to do, say, from $0\le\theta\le\pi/4$), or you need to impose a condition on $a$ and $b$ such that this discriminant is negative, and you get only complex conjugate values for $z^2$.
So, let's suppose that is the case (and with the factors above, hopefully it will be a bit easier to find out when that happens): the roots of the quadratic in $z^2$ are complex conjugate pairs. That means $z^2=r e^{\pm i\theta}$ for some positive real $r$, and $0<\theta<\pi$. Finally, we have the four poles located at
$$z=\sqrt{r} \, e^{i\theta/2}, \; \sqrt{r} \, e^{-i\theta/2}, \; -\sqrt{r} \, e^{i\theta/2}, \; -\sqrt{r} \, e^{-i\theta/2}.$$
The first and last of these, if you think it through, will have positive imaginary part.
So, what is $\theta$? Well, we know that
$$\tan(\theta)=\frac{\text{Im}(z^2)}{\text{Re}(z^2)}.$$
Work out what that is, and you should be on your way. Don't forget to make sure $0<\theta<\pi$ by adding or subtracting $\pi$.