This is a common mistake, particularly with A-Level math students. The second derivative gives you information about the rate of change of the derivative, or the curvature of the curve. Now, if the second derivative at some point is positive this means that the curvature at this point is concave up, like the shape of y = x2. Equally, if the second derivative is negative at some point, this means that the curvature is concave down, like the shape of y = - x2.
Now, a point of inflection means that the curvature of the curve has change, e.g. from concave up before the point, to concave down after the point. Now, if we have some value of x, say x = a such that f''(a)=0; then it is quite possible that this is a point of inflection. However, to be certain of this we need to look at the curvature either side of the point. I.e. we need to take [itex]f''(a - \delta)[/itex] and [itex]f''(a + \delta)[/itex] where [itex]\delta[/itex] is a small positive number. If the sign of the second derivative changes from before x=a to after x=a, then we have a point of inflection.
I hope that made sense.