Well - it is correct for a restricted class of functions. Saying [itex]\int f'(x)dx =f(x)[/itex] presupposes that f(x) is differentiable (otherwise the expression is meaningless). The other way around ([itex]\frac{d}{dx}\int f(x)dx[/itex]) allows for a larger class of functions (the Riemann-integrable functions).
Going to an even larger class of functions, we have the following theorem of Lebesgue:
If φ(x) is a summable function, its indefinite integral [itex]F(x)=\int_{a}^{x}\phi(t)dt[/itex] is a continuous function of bounded variation and it has almost everywhere a derivative equal to φ(x).
Observe the "almost everywhere" clause which is typical for all integrals based on measure theory. Lebesgue also proved a theorem about the other direction:
The derivative φ(x) of an absolutely continuous function F(x) defined on the closed interval [a, b] is summable and for every x [itex]\int_{a}^{x}\phi(t)dt = F(x)-F(a)[/itex].
Observe the restriction on φ(x)!