"1" is just a label, it could be anything. Consider this model of Peano arithmetic:
"1" is a triangle. If x is a polygon, Sx is a polygon with one more side than x; otherwise, Sx is a pink unicorn. x = y iff x and y are polygons and x and y have the same number of sides. For example S(S(1)) (that is, "3") is a pentagon.
This works perfectly well -- all the Peano axioms can be made to hold in this system, even though the underlying objects are not "numbers" in any normal sense of the word. If we went through all the usual definition we would find that x + y would be defined as a polygon with two fewer sides than the total number of sides in x and y.