The book "Modern differential geometry for physicists" by Chris Isham is a very good place to start. It explains what you need to know about bundles, Lie group actions, etc. Unfortunately it doesn't go all the way. It doesn't cover integration on manifolds and the YM Lagrangian.
If you don't know integration on manifolds already, the book "Introduction to smooth manifolds" by John M. Lee is a good place to learn it.
I don't know what's the best place to continue after Isham. The one that looks the best to me (judging only by the table of contents) is "Topology, geometry and gauge fields" Gregory Naber. It seems to cover a lot of the stuff that's covered in Isham too, so you may not even need Isham (but it can't be a bad idea to get both). A few other books that look interesting:
"The geometry of physics" Theodore Frankel
"Gauge Fields, Knots and Gravity" Baez & Muniain
"Differential geometry and Lie Groups for physicists" Marián Fecko
There's also a review article called "Preparation for gauge theory" by George Svetlichny. I found it too hard to read, so I can't really recommend it for anything other than the quick intro to group actions on the first few pages.