We will never know for sure simply because there is no record of any reason. The most likely scenario is that it simply did not occur to Nobel. He wanted to award the most important “discovery or invention” and selected five areas out of which four were very close to his own interests and the fifth (medicine) of obvious benefit to mankind in Nobel’s mind. As an applied engineer, it is likely that he did not consider mathematics to be of direct benefit or to encompass discoveries or inventions in the sense he meant it.
Swedish mathematicians Hörmander and Gårding wrote a
peer reviewed paper on the subject of why there is no Nobel prize in mathematics and effectively debunked the idea that it was because of an affair. However, the story sounds juicy and is therefore retold without reasonable source criticism.