Some elements' electrons are more tightly bound to the atom than others. Oxygen, fluorine, chlorine, and other elements in the top right of the periodic table hold on to their electrons very tightly. Elements on the bottom left, like francium, cesium, and barium are almost giving away their electrons. The more loosely the electrons are bound to the atom, the more conductive the element is.
At the same time, the alkali metals can only give away one electron and the alkaline Earth metals can only give away two because the remaining electrons are in what is called a closed electron shell. Elements like in the copper group (copper, silver, and gold) have the most electrons to give away. This means that even though one electron isn't as bound as the the first two groups, they make up for this in sheer number.
The freer the electrons are, the more they behave like a gas (or liquid, as in "electron sea") inside the metal, and therefore the better they can flow through material. The more electrons, the greater that flow can be.
So far, we've only looked at the chemistry of conductivity. There is also a lot of physics involved as well. This is where that lattice you were talking about comes in. The atoms in a pure metal are uniform and packed really tightly. Atoms of different elements are different sizes. This causes irregularities in the lattice, and disrupts the smooth flow of electrons. There are also phenomena that lead to an increase of conductivity, but this is an upper level undergraduate/graduate topic, not very easily comprehended by even professional researchers, let alone high schoolers. This leads to some counter intuitive phenomena like superconductivity and semiconductivity.