There's a key difference, though. The article puts it this way: "there is an arbitrariness in the choice of which particle's mass to use", whereas only one charge can be used to define the fine structure constant. What this really is is a manifestation of the fact that the fine structure constant is genuinely dimensionless, whereas there is no genuinely dimensionless coupling constant for gravity, at least not as we currently understand gravity. In QFT terms, we should not expect the coupling constant for gravity to be dimensionless since the QFT of a massless spin-2 field is not renormalizable; whereas we should expect the fine structure constant to be dimensionless since QED is renormalizable.
In terms of looking for variation, if we found evidence of variation of, say, the dimensionless gravitational coupling constant defined using the electron mass, we wouldn't know whether that was evidence of gravity itself varying or of the electron mass varying, since, as the article notes, the electron mass is due to the Higgs mechanism (according to our best current understanding), which is independent of gravity. There is no such ambiguity with regard to the fine structure constant: it is a "pure" coupling constant for electromagnetism, with no other mechanism involved, so if we found evidence of it varying over time, there would be only one possible interpretation.