You're right! You can look at how Spivak defines complex numbers, as a pair ##(a,b)## along with the definitions ##(a,b) + (a,d) = (a+c, b+d)## and ##(a,b) \cdot (c,d) = (a\cdot c - b\cdot d, a\cdot d + b \cdot c)##. The ##x+iy## notation is recovered by setting ##i = (0,1)##, and ##(a,0) = a##. There's nothing very strange about this at all. And imaginary numbers are just a subset of ##\mathbb{C}##, of the form ##(0,b)##.
Historically, mathematicians have a tendency to be skeptical about new developments, hence why "imaginary" was introduced as a bit of a diss.