The problem is all about what an indefinite integral is. It represents area under the curve drawn by graphing the integrand alright, but you just don't know the "bounds" of your domain, in other words, which part of the area you are actually going to choose.
For example, you can pick $\displaystyle \int \frac{\mathrm{d}x}{x}$ to be the area under $y = 1/x$ for $x < 0$. In that case, you'll get $\log(-x)$ as the result of integration. So $\displaystyle \int \frac{\mathrm{d}x}{x}$ is really defined piecewise as $\log(x)$ if $x > 0$ and $\log(-x)$ if $x < 0$. At $x = 0$, the integrand $y(x) = 1/x$ blows up, so it's none of our concern.
Note that $|x|$ always "positifies" any real number, i.e., for $x > 0$, $|x| = x$ but if $x < 0$, $|x|$ multiplies by a minus sign to make the number positive, i.e., $|x| = -x$. But this is exactly what is happening to the argument of $\log$!
So the best way to make the piecewise function work up in a single equivalence is to say
$$\int \frac{\mathrm{d}x}{x} = \log|x| + C$$
EDIT To make this more convincing, consider $(\log|x|)'$. This is NOT $1/|x|$. Note that $y = |x|$ is also a function, and $\log|x|$ is thus a composition of two functions. To differentiate this, you need to apply the chain rule, which gives $(\log|x|)' = (\log|x|)'_{|x|} \cdot (|x|)'_x$. It is clear that $(\log|x|)'_{|x|} = 1/|x|$, so we just need to work out the other part. $y(x) = |x|$ is everywhere differentiable except $x = 0$, but there $\log(x)$ blow up, so we don't need to care about it. By definition, $(|x|)'$ is $1$ if $x > 0$ and $-1$ if $x < 0$, i.e., this acts like a "sign" of the real number $x$. Let's call it $\text{sgn}(x)$.
Hence we have $(\log|x|)' = \text{sgn}(x)/|x|$. But then we are multiplying the absolute value of the real number $x$ by it's sign, which gives us back the original real number $x$. Hence,
$$(\log|x|)' = \frac1{x}$$
And by definition of antiderivatives, the equality above is verified.
