Have you ever seen plots like this one?
http://en.wikipedia.org/wiki/Image:EfieldTwoOppositePointCharges.svg
Imagine that the positive and negative charges on this plot are actually the positive and negative terminals of a battery. For every point in space around the battery, you can draw a little vector indicating the magnitude and direction of the electric field. The electric field theoretically extends outwards to infinity.
When you put a conductor in the electric field -- by connecting the two terminals of the battery with a wire, say -- the field lines change, but that's not very important at the moment. Just recognize that the electric field exists inside the wire now as well, in much the same way that it existed in the air before the wire was introduced. The difference now is that the wire conducts electrons very well, while the air did not. The electrons are now able to flow from an area of "high" to "low" electrical potential. (Yes, "conventional current" has all the polarties reversed, but I don't want to get into the semantics of it, which I why I put those words in quotes. You can think of electrons moving one way, or mythical positive charges moving the other way, and the effect is the same. High potential for a negative charge is low potential for a positive one, but, in either case, the situation is pretty much analogous to a ball rolling down a hill.)
The electric field, now present in the wire, motivates electrons to slowly drift down the wire, from the negative terminal to the positive terminal.
- Warren