The theory is simple. You hang a mass from a spring and bring another mass up close below it. The spring will stretch slightly and you use Hooke's law to measure the force. Then you try again with a smaller mass and you'll see a smaller stretch.
The problem is how weak gravity is. Two 1000kg masses with their centers separated by 1m (think about the size of a 1000kg mass) produce around a 0.07mN force. Any spring balance sensitive enough to detect this will snap when you attach a 1000kg mass. If you try a smaller mass then the force goes down (so harder to measure) and (in a double whammy) the system becomes extremely sensitive to thermal noise - draughts, Brownian motion, cars passing in the street, earthquakes thousands of miles away, etc, etc.
Measuring g between two known masses is the way we calculate G. The pros take years to make a measurement, and they frequently disagree. You could be decades at this and get nothing but noise - sorry.
Easier: track planetary motions and confirm that the orbits match predictions from theory. More maths and more indirect, but practicable for the dedicated hobbyist, I would think, given that Kepler and Newton have already done the heavy lifting.
