Literally, the quote comes from scientia potentia est, by Sir Francis Bacon, and it is essentially an argument that heresy and atheism are "rebellion and mutiny" against his god and therefore stupid, and stupidity is a weakness of character.
So basically it pretty much comes from a Kirk Cameron -esque dig on atheists, but in Latin.
In the colloquial meaning of the phrase, I agree, and it's unfortunate that I can't use the Latin phrase as a personal motto due to its original meaning. Knowledge IS power, especially in the modern world. If you're smart, you can get a better job: you get hired as a doctor or engineer because you know things about medicine or technology, and money gives you the power to do things you want. If your country has better scientists and engineers, you will have better technology, and that means everything from more comfortable living to more powerful weapons of war.