The fillers and coatings answer is most of it. Vitamins and some other nutrients have color, but they are usually in such small abundance in a pill as to not add much color. (Except the horse pill sized supplements that contain everything remotely like a human micronutrient. ) Many pharmaceuticals are whitish anyway. We are assuming they are powders or crystals - for the example below.
Suppose you take, um, 20mg of spironolactone - a diuretic. It has density approximately equal to table sugar. So, get yourself a magnifying glass and some tweezers (forceps). Count out 10 grains of granulated sugar. Put them in a neat pile. Can you even see it from 2 meters away? If you dropped a pill that small would you ever find it? Try getting just a single one of those "nanopills" out of a bottle.
There are good reasons for fillers - you can see the most obvious ones already. Us. People who take the pills. Coatings and shapes identify pills to consumers of those pills. 'I have to take two pink ones, a blue one, and the white long skiny one'
Fillers and coatings can affect absorption of the drug, timing it to release slowly, or release in the intestine, or withstand stomach acid. Not mention help fumble fingered people with decent sized pills. All this from pills almost completely made of fillers and coatings.
Pills also have small mnemonics in/on the surface so Pharmacists can identify them. The formulaies they have come with nifty pill pictures. To decode the pill. Try that with a nanopill.
