PeterDonis said:
But in these coordinates, while we would certainly expect the particle to be deflected, we would not, it seems to me, expect the system of large masses (the solar system in this case) to be disrupted. We would expect its configuration to be basically the same after the flyby as before--because in this frame the particle has negligible effect on the spacetime geometry.
However, a flyby of either a one solar mass object or a two solar mass object (meaning, the total energy of the object taking into account that in the solar system frame it is moving at a tiny smidgen less than the speed of light) through the solar system would be expected to disrupt the solar system, not just deflect the object.
So we still have a disconnect here: in one frame (the object's rest frame), we expect no disruption of the solar system, but in another frame (the solar system rest frame), we do. So the intuitive reasoning described above must be wrong in at least one frame.
What happens in the spaceship frame did has me puzzled a bit. It appeared at first glance to contradict the paper, but it turns out there is no conflict. I see that Vanadium made similar comments.
I will note that the paper I cited, and my analysis below is for dust, not planets, so I will not address that point that you raised.
However, what happens for the dust case in the frame of the spaceship is interesting, and I think I have a handle on it. The ultra-relativistic dust flies by the spaceship, which has some very small mass, so the dust is deflected only slightly in the frame of the spaceship. Essentially, it should be deflected by twice the Newtonian deflection due to the small, but non-zero, gravity of the spaceship. It's well known that light deflects twice as much in GR as in Newtonian physics, and the difference between the geodesics of light and the geodesics of ultra-relativistic particles is negligible.
Thus we expect in the space-ship frame that the dust will be deflected by some small angle ##\theta##, winding up with a velocity in the longitudinal direction of ##\beta \cos \theta## and a transverse velocity of ##\beta \sin \theta##, where we can use the usual GR formula to compute ##\theta##.
However, to transform from the space-ship frame to a frame where the dust was initially stationary, we essentially need to do a relativistic velocity subtraction.
The velocity subtraction formula with a transverse and longitudinal components is a bit messy, wiki gives it in
https://en.wikipedia.org/wiki/Velocity-addition_formula
The relevant formula is that the longitudinal direction is "x" in the wiki analysis, and the transverse direction is "y".
The formula wiki gives is then
$$ u^\prime_y = u_y \frac{\sqrt{1- \beta^2}}{1-\beta \beta \cos \theta}$$
which with ##\cos \theta ~ 1## should basically mean the transverse velocity in the dust frame should be larger by a factor of approximately ##\gamma## than its small value in the spaceship frame.