It's the same thing about currents: the heavy bypass caps should shunt the 60Hz 60/120 Hz currents around the load/signal circuitry. So picture the 60/120 currents going in the top and coming out the bottom of the cap and then going back to the rectifiers and regulators. You want the signal circuitry to "see" the stable cap voltage not the bypassed current. So the grounding of the signal circuitry should be nailed to the bottom of the cap; otherwise it will be exposed to the 60Hz current. Been there done that :) This exposes you to some other problems, like ESD so it's not a cure-all but one factor to balance.
As far as the scope goes:
1) Plug it into the same 115vac wall socket as the circuit. This will go a long way towards mitigation of scope ground interference. There was a story about different instruments plugged into different outlets in the 60's at a hospital, and then the janitor plugging in a vacuum cleaner in the hall; sending a current through the patient's heart. Not a good scene; but memorable. Know where your kids and currents are! In any case, the scope ground will be pumping current out unless it's a very good (and more expensive) design so try to make sure that where you attach the scope ground to your circuit won't interfere with your circuit operation. This can work both ways; there are a couple of tests that can be done to make sure you are seeing the truth on the scope. One important one is to put the scope probe to the point where the scope ground is connected and look at the residual (with your circuit active); this a simple "common mode" test of how much you can believe readings. Unfortunately, it doesn't cover all possible things that can cause false signals.
Try-- http://cp.literature.agilent.com/litweb/pdf/5989-7894EN.pdf
Common mode is item 6.
Agilent has been in business for a number of years and know's what they are talking about. (A few missteps but whose counting?)
The point is: keep a critical attitude about the whole schmear, everything connected to your assembly, when trying to dig out interference in particular 60/120 Hz.