Here's another way to look at it: Make a clock by bouncing a light pulse back and forth along the y-axis between two mirrors separated by a distance D. The mirrors are at rest in your frame; their x, y positions are not changing. Say that each cycle of the light pulse takes one second of time in your frame.
Now consider another pair of mirrors, still separated by D in the y direction, but moving along x. Because the speed of light is the same and the diagonal distance is greater, you will measure a time longer than one second for each cycle between the moving mirrors.
However, an observer traveling with the moving clock will measure one second for each cycle, because he measures the same transverse distance D and the same speed of light. You therefore measure a dilation of time compared to what he measures. And it doesn't matter what direction he is moving (along x) or how fast. The diagonal distance gets longer when the clock is moving.
This applies equally well to the traveling observer. He measures the light pulse cycle time in his clock as one second. But looking back at your frame, which is moving with respect to him, he sees a diagonal distance traveled by the light pulse in your clock. That longer distance gives a time dilation in your frame as seen by him.
The use of a transverse length, along y in this example, is the key to comparing measurements among moving frames. Transverse lengths are agreed upon by all observers. [Note that the proper interval ds acts like a transverse length.]