I think the answer to that question is less obvious than some people might think, especially if they were to respond with "Those are the field lines." (??)
Here's what I think: the filings have a certain amount of friction with paper, which is what prevents them from bunching up the way you describe - I agree that is what they would do in the absence of any friction (or other) opposing force. But - even though they cannot easily slide across the paper, they can more easily rotate. Each little filing piece will become magnetized and develop N and S poles, and thus experience a torque from the external field. This torque will go to zero when the filing rotates so that it is aligned with the direction of the magnetic field vector at that point, so that alignment is an equilibrium configuration. When you have a whole bunch of filings, they all become magnetized and the effect is magnified, that is, they'll line up head-to-tail and give the appearance of lines.
I have no idea if this is what's really happening, but it's my best guess! ;-)