Men generate sperm pretty much over their lifetimes once they reach puberty.
Women's ovaries have oocytes (eggs) that are in early oocyte formation. These oocytes are there in the ovary before the birth of the female baby.
In simple terms this means that men's sperm is formed anew, over a short period of time. Oocytes in women are "stuck" in the middle of egg formation (meiosis) until the egg is shed. In general, genetic damage or problems in meiosis are much more common as women age.
Meiosis splits pairs of chromsomes during the process of making eggs or sperm. Human eggs/sperm have 23 chromosomes. This number, 23, is called N for humans. The cells in our bodies all have two copies of each chromosome, so these cells are 2N.
When an egg (or sperm) is formed and the pairs of chromosomes do not split correctly, or "mess up", you get an extra chromosome in one egg, or you are entirely missing that chomosome in another egg. An extra chromosome may not be fatal to a newly fertilized embryo, but missing all
of one type of chromosome is normally very fatal. So, the eggs with extra chromosomes have a small chance to fetilize, develop, and become a new baby.
As women age they produce an increasing percentage (small) of eggs with these kinds of problems.
Down's syndrome is caused usually by Trisomy 21 - three copies of the number 21 chromosome. This is meiotic failure. Down's also manifests in people with 15-21 translocations - pieces of chromosome #21 are swapped with chromosome #15. This is very rare, but is also a meiotic failure problem.
Generally, most of the risk of children with these kind of abnormal chromosome problems is attributed to female age at conception than to male age at conception.
There are other conditions that have nothing to do with age, but may be related instead to family history or to exposure to mutagens.
See:
Thompson & Thompson Genetics in Medicine, 6th Edition by Nussbaum, et al. 2004