Two offending classes of compounds are phthalate esters and synthetic musks.
I find most perfumes and colognes irritating, and most middle-aged and elderly women use way too much perfume probably because they lose their sense of smell and so overdo it. I have no problem with ethanol.
Anyway, Professor Thomas Kearns has published a book: Environmentally Induced Illnesses : Ethics, Risk Assessment and Human Rights, in which is discusses the problem of chemical sensitivity.
"The fragrances in aftershaves, perfume, hair spray, cologne, detergents, shampoos, and many other products we encounter every day contain solvents and neurotoxic chemicals which cause severe physical reactions in those people who have become sensitized to them. These reactions can be mild, such as a slight headache or a sudden inability to think or concentrate, which might not even be recognized as the result of exposure to fragrance; or can sometimes be severe, such as a migraine or an asthma attack, and can interfere seriously not only with study but even with the ability to breathe. The National Academy of Sciences has estimated that approximately fifteen percent of the population suffers from some level of chemical sensitivity." T. Kerns, North Seattle Community College, Letter explaining the Policy on Indoor Air Quality & Chemical Sensitivity