I'll give my view on this. Keep in mind, there usually no set definitions, much of philosophical reading is highly contextual.
Parsimony - The creed to simplicity. This is not always justified, but the basis is that the idea can be subjected to probabilistic analysis which generally favours fewer terms due to the multiplicative nature of probabilities.
Reductionism - Can be best understood by contrast to reification: the making of the nebulous to be concrete. In reverse, reductionism is used often to imply the making of the concrete into postulates.
From this, you should find that the impetus is different. In parsimony, a purely probabilistic consideration is being made. Reductionism seeks to find the basis of a concept. As such, parsimony can be applied to a reductionist argument and reductionism can eliminate non-essential postulates for parsimonious reasons.
At least this is the lie I've been living with for some time now.