A better analogy for voltage using a constant gravity source such as the gravity near the Earth's surface is height. Gravitational potential reasonably near the surface of the Earth equals g h, where g = 9.80665 m / s^2 and h = height in meters. Treating g as constant, the gravitational potential is a function of height. So if an object is above the surface it has positive gravitational potential, and if an object were below the surface such as inside a hole it would have negative gravitational potential. You could also define gravitational potential to be zero at the top of a 100 meter tower, so that gravitational potential equals g (h - 100). This would result in negative potential for h < 100 and positive potential for h > 100.
The units for gravitational potential is joules / kg, while the units for electrical potential (voltage) is joules / coulomb.