The point is that, under a carefully chosen time-dependent transformation involving a boost and rotation of your coordinates, the rigid body can be made to perform any possible motion.
Anyway, suppose you did settle on a particular frame. Now, select any point ##\mathcal{O}## at ##\mathbf{x}_{\mathcal{O}}## on the body, and let's consider how the position of another point ##\mathcal{P}## at ##\mathbf{x}_{\mathcal{P}}## on the rigid body changes between times ##t## and ##t + \delta t##. The point ##\mathcal{O}## undergoes a translation ##\mathcal{O}(t+\delta t) - \mathcal{O}(t) = \delta \mathbf{x}_{\mathcal{O}}##, and the change in the position of ##\mathcal{P}##, i.e. ##\mathcal{P}(t+\delta t) - \mathcal{P}(t) = \delta \mathbf{x}_{\mathcal{P}}##, can be described by compounding the translation ##\delta \mathbf{x}_{\mathcal{O}}## with a rotation by ##\delta \phi## about an axis ##\mathbf{n}## passing through the new position of ##\mathcal{O}##, i.e.$$\delta \mathbf{x}_{\mathcal{P}} = \delta \mathbf{x}_{\mathcal{O}} + \delta \phi \mathbf{n} \times (\mathbf{x}_{\mathcal{P}} - \mathbf{x}_{\mathcal{O}})$$Use the Physicist's trick of dividing by ##\delta t##,$$\frac{\delta \mathbf{x}_{\mathcal{P}}}{\delta t} = \frac{\delta \mathbf{x}_{\mathcal{O}}}{\delta t} + \frac{\delta \phi \mathbf{n}}{\delta t} \times (\mathbf{x}_{\mathcal{P}} - \mathbf{x}_{\mathcal{O}}) \implies \dot{\mathbf{x}}_{\mathcal{P}} = \dot{\mathbf{x}}_{\mathcal{O}} + \boldsymbol{\omega} \times (\mathbf{x}_{\mathcal{P}} - \mathbf{x}_{\mathcal{O}})$$As it turns out, the point ##\mathcal{O}## that we chose need not even be "on" the rigid body, the only requirement is that it is fixed with respect to the rigid body. Furthermore, you can show (try it!) that whichever such point ##\mathcal{O}## you choose, we get the same angular velocity vector ##\boldsymbol{\omega}##. To actually solve mechanics problems, you either take ##\mathcal{O}## to be the centre of mass [for general motion], or the point on the body fixed in the lab frame [for pure rotation].
You can also choose a particular point ##\tilde{\mathcal{O}}## such that the transformation between ##t## and ##t + \delta t## is carried about purely the rotation ##\delta \phi \mathbf{n} \times (\mathbf{x}_{\mathcal{P}} - \mathbf{x}_{\tilde{\mathcal{O}}})##, i.e. with ##\delta \mathbf{x}_{\tilde{\mathcal{O}}} = 0##; this is the instantaneous centre of rotation. [Although, for motion with no rotation, this will be at infinity...]