Looks to me like the problem starts from remembering the acceleration, or looking at the printout of the accelerometer, or any other record keeping device or system that reveals a past real period of acceleration... prior to achieving the inertial reference frame.
... and the problem then comes from thinking that you were "still" before the acceleration, but now are "moving" as a result of it... shades of absolute space.
This is not a technical answer, but an inertial reference frame is "still" when it is yours no matter what history of previous accelerations might have been. In essence, an IRF has no "mechanical history" of accelerations (no lingering, ontological, causative, or otherwise potent effects). Another way to say it is that any and all possible acceleration histories are "equal" in their nonexistent effect after an IRF has been established - all IRF are physically identical despite any historical differences with respect to acceleration.
If you think about the implications of this, you will see that it never makes sense to say, "My IRF is moving and I know it because of this piece of evidence of past acceleration..."
As phinds often says, sort of, "You are moving and not moving at all possible speeds, depending on who is measuring and their relative motion, but do you feel any different?", or something like that... :)
Which means that to somebody somewhere in relative motion to your IRF, they might account for your relative motion to themselves by suggesting any and all possible combinations and configurations and durations of acceleration history to account for your observed relative velocity... each acceleration history unique, and all acting prior to your same final relative motion.