Adjective phrases and adverbial phrases are types of prepositional phrases that act as adjectives or adverbs.
"Mary and Martha took her children to the park" is truly a reference error. There is a preference for "Martha" but it is not clear whose children they are. There is also, however, a set of implicit rules for determining which reference is intended, which is the reason the vast majority of literate people--i.e. respected authors--make certain kinds of so-called "reference errors" continually yet nevertheless understand one another easily and unambiguously. I believe the reason some grammar textbooks deny this set of rules is for simplicity.
To demonstrate this, I searched for Charles Dickens on Google, and the first work of his I clicked on the first site that came up was A Christmas Carol. The first thing I read under that heading is his introduction:
Charles Dickens said:
I have endeavoured in this Ghostly little book,
to raise the Ghost of an Idea, which shall not
put my readers out of humour with themselves,
with each other, with the season, or with me.
May it haunt their houses pleasantly, and no
one wish to lay it.
Their faithful Friend and Servant, C. D. December, 1843.
The sentence "May 'it' haunt their houses pleasantly, and no one wish to lay 'it'" contains two so-called "reference errors." However, there is no ambiguity and nobody would claim Charles Dickens is a poor writer.
One of these implicit rules is that nouns in adverbial phrases count for the most recent noun, and nouns in adjective phrases--mere attendants to and parts of other nouns--do not count. A reference to a noun in an adjective phrase can be forced through context, generally with some degree of awkwardness, but when no context forcing is present, the default is to ignore the adjective phrase for the purpose of reference but not ignore the adverb phrase.
For example, from the same story (do a search to find the place):
"But I am sure I have always thought of Christmas time, when it has come round"
The "it" refers to "Christmas time," even though "Christmas time" is inside an adverb phrase.