I feel a compelling need to point out it's x2/2 actually. Regardless.
If you have f(x) = o(p(x)) for some f and p, it means that as x goes to zero, f goes to zero faster than p, i.e. f/p -> 0 as x goes to zero. This is meant to indicate that near zero something is so small, it's going to vanish when you take a limit to zero.
O is the opposite. f(x) = O(p(x)) means that as x goes to infinity, f is bounded above by p, i.e. there is some constant C such that f(x) <= C*p(x) for all large x.
So in your Taylor series example, I don't agree with what's written (assuming you meant little o as you stated in your second sentence) as the remainder is
[tex]x^3/6 + ...[/tex] and when you divide through by x3 you get [tex]1/6 + x*(stuff that's at least constant)[/tex] which doesn't go to zero. It IS o(x2 as when you divide through by x2 you get
[tex]x/6 + x^2/24 + ... = x*(stuff)[/tex] and that goes to zero as x goes to zero