Imagine taking a long straight length of conductor and placing this in a uniform E field (let's say to the right). What would happen? Initially electrons would begin to move in the opposing direction of the E field, and as a result the leftmost side of the bar would gain a net negative charge and the right hand side would gain a net positive charge (due a lack of electrons). This process cannot continue forever, at some point the electric field due to the net positive and negative charges at each end of the bar, will cancel out the external E field, restoring equilibrium and ceasing motion of the electrons. Thus, no more current flows.
With respect to a negative test charge, the -ve end is at a higher potential than the +ve end. If you turn off the external E field, electrons from the -ve end will move to the right to the lower potential (the voltage has been "dropped" across the bar). Unfortunatley in this example the current will be transistory, not steady, and therefore useless for an application.
Batteries (or other simpler sources of emf like moving a closed wire through a uniform B field) work by causing electrons to move against the potential (i.e. go from where there is already a surples of positive to where there is a surplus of negative). In fact the simplest source of emf is to mechanically (on a kind of treadmill) move the negative charges to the higher potential (w.r.t. to the neg charges). This is what supplies the energy (in the example above of the bar, emf could be generated by me "picking up" electrons and moving the from the right to the left.
A good analogy is that of the water fountain, water flows out and falls to lower gravitational potential (this is the electrons flowing naturally from the -ve end to the +ve end, when the external field is switched off). But if this was all there was to the fountain, it would be a very short display (transitory current). What is needed is a way to take the water from the lower potential (ground level) to a higher potential (top of pump), against the gradient. This is were the pump comes in (analogue of the emf). The pump mechanically does work to move the water "up the potential hill", so that this potential can then be "dropped", i.e. the water releases that potential in the form of kinetic energy and flows to bottom of fountain.
One thing to note is that real sources of emf have an internal resistance, this is the resistance encountered for example within the battery, when moving the electrons from +ve to -ve (in the unnatural direction). Therefore the potential that can ultimatley be dropped across the circuit is slightly less than the emf (note emf is a misnomer, as its not a force but expresses the amount of work required to climb the potential hill). V=emf-Ir, where r is the internal resistance. Noting from ohms law that V=IR (R is the usual circuitory resistance), we arrive at the expression IR=emf-Ir => emf=I(R+r).
In the fountain analogy the internal resistance could be thought of as something like friction, by which I mean the pump does x amount of work to raise the water to the top of the fountain, but not all of that work will go into raising the water's P.E. some will be dissappated because of friction etc. So the amount of potential that can be dropped by the water will be slightly less than x. V=emf(of pump)-energy_dissappated.