I think the problem is that you are taking interpretations of the derivative and the integral (which are inverse operations) for those things, themselves.
Integrals can be understood as the area under a curve, but that doesn't mean that they are the area under the curve. For example, if you integrate the velocity of an object, you'll get its displacement. Displacement is in meters, not meters squared - so how does that jive with the integral as area under a curve? The answer lies in the fact that the "height" of the curve (i.e. the y-axis) is in m/s and the "base" (i.e. x-axis) of the curve is in seconds, so when you multiply base x height to get area, the units of time vanish and you get an "area" whose units are in meters ... This isn't as strange as it may sound at first. The key is that the idea of an integral as an area is an interpretation
ps. to make this clear: consider the language of your question, itself: you're identifying the derivative with the tangent, when in fact that isn't exactly the interpretation of the derivative commonly used. The derivative is the slope of the tangent line - not the tangent line itself. So the question would be re-phrased as "how are the slope and the area inverses", again they aren't really. Now if you think of y and x as both being measured in meters, then the slope as no dimensions (it's just a number) whereas the area has units of meters squared ... So in first case (taking the derivative) we start with meters and end with no units (just a number), but in the second case we start with meters and end with meters squared. Perhaps that analogy helps clarify how a slop and an area are "inverses" - ?