You made the classic mistake of thinking that it's motion of galaxies moving away from each other into existing space. It's not. It's not space was there, and boom stuff was created and now it's flying away from each other like an explosion. It's spacetime itsself that is expanding.
You idea that it could just be the mass that was created violates the 1st law of thermodynamics, which if the Universe was existing before hand follows the same phyical laws that govern us. Not only that, but if just the mass was created in the existing universe then there would be a definitive centre that could be found. Also if all the matter were created in 1 single spot, gravity would dominate and it would collapse back in on itsself.
Those are three of the big problem with the idea that it was just the 'stuff' that was created within the universe, not the universe itsself.
There are also many other peices of evidence that point to a small dense hot beginning to the universe.