It is the difference between an
improper integral and a
principal value integral.
In the former, we need to evaluate
##\lim_{x\to0} \int_{-1}^{x} \frac{1}{t} dt + \lim_{y\to0} \int_{y}^1 \frac{1}{t}dt##.
Notice that x and y are independent of each other. If you take the limit of x faster* than y, you will evaluate ##-\infty##. If you let y converge faster than x, you will evaluate ##\infty##. These don't match hence the integral is undefined.
With PV integrals we need eavluate
##\lim_{\epsilon \to 0} \left(\int_{-1}^{-\epsilon} \frac{1}{t}dt + \int_{\epsilon}^{1}\frac{1}{t}dt \right)##
Notice how they converge from both sides at the same rate. Hence the two integrals always cancel leaving zero.
Unless you explicitly state principal value, the default is to assume improper as it is more useful in applications.
* Aside: you should try to think of limits as a process that converges to anything, but in the this example it works.