How exactly do you mean think? Do you want the definition usually given, or the intuitive concept of subtraction. Normally we take it as an axiom that every x has an additive inverse y such that x+y=0, and we then think of subtraction as the addition of the additive inverse. Or in more ordinary terms we define -x to mean the additive inverse of x, and define x-y to be equal to x+(-y). Note that you would have to prove the uniqueness of the additive inverse for subtraction to be well-defined, but that's fairly trivial.
Of course all correct definitions are equal, so you can define it however you want. If you have defined -1 (the additive inverse of the identity), and multiplication between arbitrary elements then you can define x-y to mean x+(y*-1) and it would be just as valid as the first definition I gave (in fact they can be proven equivalent for most definitions of multiplication).
The problem with you approach is that for some groups we haven't defined a multiplication operator, we have defined several multiplication operators, or it doesn't behave like a normal multiplication operator. However for fields and rings the two definitions are equivalent.
I just realized that you may be only thinking of elementary algebra. If you are only working within the real or complex numbers then the definitions are equivalent as well, so yours isn't inferior in any sense.