Actually, the center of gravity is not where the whole weight acts on. The force of gravity acts on each particle in the object at roughly the same magnitude. It's merely the point where if you apply a force that is inline with it, no torque is generated.