I feel there's some missing context.
It looks like you're triny to solve [tex]
m\ddot x + 2c \dot x + m\omega^2 x = 0.[/tex] The solution to this is [tex]
x(t) = Ae^{\lambda_{+}t) + Be^{\lambda_{-}t[/tex] where [tex]
\lambda_{\pm} = - \frac cm \pm \sqrt{\frac{c^2}{m^2} - \omega^2}.[/tex] Now iif [itex]c^2 \geq m^2 \omega^2[/itex] then that's fine as it stands, because the quantity under the square root is positive and you'll get a real result.
Otherwise, the quantity under the square root is negative and you get a complex result, which you can write as [tex]
\pm\sqrt{ \frac{c^2}{m^2} - \omega^2} = \pm\sqrt{-\left(\omega^2 - \frac{c^2}{m^2}\right)} = \pm i \sqrt{\omega^2 - \frac{c^2}{m^2}}[/tex] where [itex]i^2 = -1[/itex]. Thus if [itex]c^2 < m^2 \omega^2[/itex] you have [tex]
\lambda_{\pm} = -\frac{c}{m} \pm i \sqrt{ \omega^2 - \frac{c^2}{m^2}}.[/tex] For some reason the factor of [itex]i[/itex] in front of the square root is missing from what you have posted.