The "Fabric" part of the fabric of space-time is just an analogy. Space-time simply says that we have space, where we measure distances with rulers, and time, which we measure with clocks.
Before relativity, we thought space and time were separate phenomenon, but after relativity, it became clear that space and time were closely related and should be regarded as a unified entity.
The "fabric" part of space-time is typically used in other analogies, typically related to cosmology and the expanding universe, which is compared in the analogy to a balloon being inflated.
The mathematical entity that describes space-time is the metric. The metric allows us to calculate lengths and time intervals, as long as we specify a curve along which the appropriate length or time is to be measured.