A pendulum swings slower at the equator because the Earth is rotating. The rotation acts to make the pendulum swing slower directly and indirectly. The direct effect is easiest to envision from the perspective of the rotating Earth-fixed frame. In this frame, a centrifugal force of [itex]r\omega^2[/itex] arises directly from the Earth's rotation. At the equator, this direct effect alone amounts to 0.034 m/s2. There is of course no centrifugal force at the poles.
For those who insist there is of course no such thing as centrifugal force anywhere, you will be forced to look at things from the perspective of an inertial frame. I leave this as an exercise to you curmudgeons. Keep in mind that the answer you get will be the same.
I also cited an indirect effect that results from rotation. The Earth's rotation makes the Earth bulge at the equator. As mgb_phs noted, this makes things at the equator further from the center of the Earth. Together, the direct and indirect effect act to make Earth's gravitational acceleration 9.780 m/s2 at the equator. This is about 0.052 m/s2 less than it is at the poles, or a 0.53% reduction in the gravitational acceleration. The direct effect accounts for about 65% of the difference.
The next leading factor in variations in gravitational acceleration after Earth rotation and the J2 non-spherical harmonic term is altitude above the spheroid. Things weigh 0.28% less at the top of Everest than they do at sea level at the same latitude. Local variation in density is a distant fourth.