The objective lens is strongly curved so that the real image formed by the objective has a large magnification. The eyepiece then views this real image. The shorter the focal length of the objective, the larger the magnification.
In reality it doesn't have to be smaller. You can combine many different combinations of focal lengths to achieve the same magnification, but a small focal length eyepiece makes for a difficult viewing experience, as you have to place your eye closer to the lens and your field of view becomes much smaller for the same magnification compared to a system with a longer focal length eyepiece and a shorter focal length objective.