Consider two capacitors in series. If they are connected to a 12 volt battery, then obviously the total voltage across both capacitors is only 12 volts, as that's all the battery can supply. Since the voltage drop must add up to 12 volts, then each capacitor must be charged to less than 12 volts individually. If they are of equal capacitance, then both would be 6 volts. Since capacitance is c=q/v, we can re-write this as q=cv to find the total charge given by this reduced voltage. Since the voltage is half what it would be if we had a single capacitor, that means that the total number of charges on each capacitor is also halved.
Remember that the number of charges moving onto or off of the two plates in a capacitor always remains equal. Since the middle two plates (one plate of each capacitor, connected by the conductor between the capcitors) are separated from the rest of the circuit by the dielectrics, no charge can leave them. In the end, the first plate of the first capacitor and the last plate of the second capacitor have equal and opposite charges, but at half of what a single capacitor would have. So this series of 2 capacitors acts like a single capacitor with half the capacitance.
Does that help?