You have
[tex]R(\omega t)= \begin{pmatrix}cos(\omega t) & -sin(\omega t) & 0 \\ sin(\omega t) & cos(\omega t) & 0 \\ 0 & 0 & 1 \end{pmatrix}[/tex]
(Rotation about the z-axis with constant angular velocity [itex]\omega[/itex])
I presume you know that the derivative of [itex]sin(\omega t)[/itex] is [itex]\omega cos(\omega t)[/itex] and the derivative of [itex]cos(\omega t)[/itex] is [itex]-\omega sin(\omega t)[/itex] and, further, that you differentiate a matrix by differentiating its components separately.
So you should see, easily, that
[tex]\frac{dR}{dt}= \begin{pmatrix} -\omega sin(\omega t) & -\omega cos(\omega t) & 0 \\ \omega cos(\omega t) & -\omega sin(\omega t) & 0 \\ 0 & 0 & 0 \end{pmatrix}[/tex]
and, of course, you can factor "[itex]\omega[/itex]" out of the matrix.
To find the second derivative, differentiate again:
[tex]\frac{d^2R}{dt^2}= \begin{pmatrix} -\omega^2 cos(\omega t) & \omega^2 sin(\omega t) & 0 \\ -\omega^2 sin(\omega t) & -\omega^2 cos(\omega t) & 0 \\ 0 & 0 & 0 \end{pmatrix}[/tex]
and factoring [itex]-\omega^2[/itex] out of that gives
[tex]-\omega^2 \begin{pmatrix} cos(\omega t) & - sin(\omega t) & 0 \\ sin(\omega t) & cos(\omega t) & 0 \\ 0 & 0 & 0 \end{pmatrix}= -\omega^2 R[/tex]