Yeah, I agree the process would be very slowly if the files were larget.
The contacts are in safe with Google service. it also applies to photo and other files with Google Drive
What your method to back up your Android phone? A whole back up or partial back up instead?
I usually back up my content individually via Android desktop manager.
How about your option?