Drakkith said:
Is it possible to explain this using just SR and moving reference frames, or do we need to invoke GR?
One's initial intuitive guess would be that SR couldn't answer this question since we are dealing with a curved spacetime. However, I think there is a sense in which we can sort of use SR to see why it's true.
First, a key fact about comoving observers (which does require GR to derive, since you need to show that the FRW metric is a solution of the EFE): all comoving observers experience the same amount of proper time between any two surfaces of constant FRW coordinate time. These surfaces are picked out by the fact that they are homogeneous and isotropic; no other family of spacelike hypersurfaces in FRW spacetime has this property.
Now consider any non-comoving observer's worldline, and ask how much proper time will elapse along it between two surfaces of constant FRW coordinate time. In standard FRW coordinates, the worldline of this non-comoving observer will have some nonzero spatial displacement, which we can consider to be purely radial. For simplicity we consider the case of a spatially flat FRW universe. The proper time along the non-comoving worldline will be the integral of the line element, which we can write as:
$$
\tau = \int \sqrt{dt^2 - a^2(t) dr^2}
$$
Of course we can't fully evaluate this integral without knowing the specific dynamics of the scale factor ##a## and the function ##dr/dt## that gives the non-comoving observer's spatial motion. However, we don't need to do any of that to see that the above integral must give a value that is smaller than the corresponding value for a comoving observer between the same two surfaces of constant FRW time:
$$
\tau_o = \int dt
$$
Technically, we do need one other premise to complete the argument: we need to recognize that the "age of the universe" is evaluated starting at some particular surface of constant FRW coordinate time. The usual convention is to use the "Big Bang" surface, i.e., the surface of constant FRW coordinate time that marks the end of inflation and the beginning of the hot, dense, rapidly expanding state that became the universe we observe.
So far I've phrased everything purely in GR terms. But of course the two integrals above look very much like the corresponding integrals for a stationary vs. a moving observer in a particular inertial frame in SR. So we could make a similar sort of argument, heuristically, in the local inertial frame of a comoving observer. But I think we would still need extra information from GR to really nail it down.