Nothing gets invalid, as long as you are carefull. You simply can't define the multiplicative inverse of 0, that's all. That's the price you have to pay for your definitions.
You ask "isnt it a reasonble criteria?".
I could give you objects that satisfy that criteria, and you would still probably be the first to say "that's definitely not a number".
On the other hand, once, people didn't know about rational numbers. Their entire world was Z, the whole numbers.
But in Z, you can't divide by 3. It's not always well defined. 2/3 isn't a whole number, and these people didn't yet accept the notion of rational numbers. So isn't 2/3 a number?
Now let me ask you another question: is the square root of minus 1 a number? Does it even exist?
Or should I ask- can it be defined?