It may be worth reading chapter XXIV in Einstein's "Relativity, the special and general theory" for some insight, see for instance
https://www.ibiblio.org/ebooks/Einstein/Einstein_Relativity.pdf
I'll do a quick outline of the argument.
Suppose you have a marble slab. If the marble slab is flat, you can cover it with tiny square tiles - in Einstein's argument, made up of little rods - because squares "tile the plane". Einstein used the term "The arrangement is such, that each side of a square belongs to two squares and each corner to four squares." which is what I mean to summarize by saying "tile the plane".
If you try this on a sphere, it doesn't work. You can't tile a sphere, not in the sense Einstein describes.
Einstein goes on to consider a thought experiment, where the marble slab is heated, and the heat expands the rods. And he supposes that every rod, no matter what material it is made from, expands or contracts by exactly the same amount, so that there is no experiment that can recover the "unshrunk" length of the rod.
Then in terms of these rods, the marble slab is no longer flat in the sense Einstein described earlier, because you can't tile it with square arrangements of rods. I'll need to go back and add something Einstein mentioned earlier, that a "square" consists of four rods of equal length, joined together so that the diagonals are equal as well. This is pretty much the way a carpenter is typically taught to build a square frame, for instance. So we've made a bit of a jump from "square tiles" to "squares made from rods" here. Rods (which can be thought of as rulers) are the fundamental entity of concern. Einstein's description is more thorough and well laid out logically than my summary, but it's also much longer. It's definitely worth a read if you have the interest and patience to do so.
So an alternative to creating a curved space with real rulers to represent our marble slab, would be to imagine how we need to distort rulers (via the temperature field) to make the space-time flat with these imagined rulers. Because our assumptions, these imaginary rulers are completely non-physical, there is no physical substance unaffected by gravity we can use to make them experimentally. Then in terms of these imaginary and idealized rulers that don't actually exist, we might be able to describe space as being flat.
I say "space" here because that's the analogy Einstein is using with the marble slab, but abstractly Einstein is applying the same ideas to space-time rather than space, he's using the analogy to make things less abstract. . And in the analogy of space-time to space, it is the principle of equivalence that says that all the rods behave the same way due to the effects of gravity (here gravity replaces the temperature field), so that there is no sort of "little rod" we can imagine that's not affected by gravity.