Space is what is measured by rulers. Time is what is measured by clocks. The same concepts are extrapolated to situations where we cannot do a measurement directly.
If mass only curved space, static objects wouldn't be accelerated by gravity.
To describe what happens a region of space-time where gravity is significant we cannot use local rulers and clocks, because different paths give slightly different measurements. We therefore choose a coordinate system which is approximates conventional space and time, then describe how local rulers and clocks behave in terms of a function of that coordinate system (for example using the "metric" which describes distances). The conventional terminology for this in General Relativity is to say that space-time is still what is described by local rulers and clocks. You could obviously switch the terminology and use "space-time" to refer to the chosen coordinate system in which case rulers would be curved relative to "space" and clocks would not exactly measure "time", but this doesn't change the physics.
When mass is present, space-time is curved in a way which is coordinate-independent, in that for example a closed space-like path constructed with rigid rulers and protractors around a mass will have a total angle that differs slightly from 360 degrees.
The expansion of the universe is not an expansion of local space, but rather that there is more space as the universe gets older. Consider the analogy of a toy car driving outwards on a flat disc of paper, where the current radius from the centre represents time and the circumference at that radius represents space. The paper is flat, so there is no force trying to move the wheels apart, but the total size of space is steadily increasing.
Contraction in special relativity is not a physical contraction, but rather a viewpoint effect similar to the way in which rotation makes things shorter in one direction and longer in another.