I'd like to add another aspect in answering the excellent question in the OP. This question, in principle leads you to the answer why we have to introduce the field concept into physics at all.
The reason is that physics has taught us that the world is described by relativistic rather than Newtonian physics, and this implies that there cannot be "actions at a distance", where indeed you'd not need a field. The paradigmatic example is Newton's theory of gravity. It says that two point masses attract each other with an inverse distance-squared law, and that this holds instantaneously, i.e., if you move mass 1 then it will instantaneously change the force on mass 2 due to gravity. According to relativistic physics, this is impossible since there cannot be causal effects (motion of mass 1) instantaneously propagating to a distant place (change of force on mass 2). So the way out, found by pure imagination and experience with experiments concerning electromagnetism by Faraday, is the field concept, according to which the interactions are mediated by fields, i.e., the force on mass 2 due to the presence of mass 1 is not instantaneous but it's a local effect, i.e., there's a gravitational field associated with any mass spreading over the entire space (mass 1 as source of its gravitational field) and mass 2 feels a force due to the presence of the field of mass 1.
Now, what has this to do with the question concerning momentum? That's also a deep concept of modern physics, namely the symmetry principles, brought into the physicists's thinking by Einstein in 1905 and into nearly final form by Emmy Noether in 1918. The idea is to address the question, which symmetries the physical laws obey, i.e., to look for transformations that don't change the form of the equations describing the dynamics of all kinds of stuff. The first thing that comes to mind is space, where symmetry has a direct geometrical meaning, and indeed both Newtonian space and special relativistic space for an inertial observer is of very high symmetry. It's the Euclidean space, and it is symmetric under translations (i.e., no point looks any different than any other) and rotations around any point (i.e., there's also no somehow preferred direction in space). Now Noether has shown that if the dynamical laws obey a continuous symmetry like the translations, there must be some conserved quantity associated to that symmetry transformation (and the math tells you precisely which quantities are these conserved ones). In the case of the spatial translations it's momentum.
Now, since special relativity, obeys translation invariance, also in special relativity one must be able to define a momentum that is conserved due to the symmetry of space under translations. Now you see another dilemma with the action-at-a-distance models: When mass 1 changes its momentum and there is just some delayed reaction of mass 2 interacting with mass 1, for some time the momentum-conservation law would be violated since particle 2 couldn't make up for the change in particle 1's momentum instantaneously, but here also the field concept helps immediately. Since the field is there everywhere, it can take up the momentum change and carry itself momentum, such that momentum conservation is always fulfilled, and that's how things work (at least in special relativity): The field is not just somehow statically associated with their sources but themselves dynamical entities that can carry energy, momentum, and angular momentum, and they obey themselves dynamical laws (equations of motion) like the Maxwell equations of the electromagnetic field.
