JD, I'm interested in how you characterize the unique role played by the gravitational field in all of this....
This whole topic of coordinate invariance versus diffeomorphism invariance is notoriously poorly explained in the literature. In fact, I am not aware of a single reference which explains it to my satisfaction.
The intuitive idea that the other fields rely on the metric tensor for their diffeomorphism invariance. What you say sounds like it might be a way of expressing what I have in mind.
I agree that if we include the gravitational field and transform everything together then the other fields typically acquire diffeomorphism invariance. And because then the geometry is represented by metric tensor the situation seems, as you put it, "devoid of background geometrical data." But suppose we consider the invariance question for some of these fields without putting gμν into the picture. That may seem a strange, unmotivated thing to do---leave out the gravitational field. It seems to me that typically (perhaps with some exceptions which you can point out) the other fields fail to have a satisfactory formulation.Diffeomorphism invariance follows when the theory is devoid of background geometrical data...
Maybe this is even obvious. Imagine using a fixed background metric, one that does not transform under diffeomorphisms, or no metric at all. How to put the other fields into covariant form?
It seems to me that the criterion you offer---"devoid of background geometric data"---could be interpreted as saying that the gravitational field is essential to the proper formulation of the other fields. That in order to transform properly the other fields must "ride" on the gravitational field. Does this make any sense to you?