This is what's called a group or Lie-algebra representation. The rotations are in this case represented by a two-dimensional complex Hilbert space with the rotations represented by unitary ##\mathbb{C}^{2 \times 2}## matrix. Now in quantum theory the vector representing a pure state is not unique, but you can always multiply it with a phase factor. Thus the rotation group is here represented by its socalled covering group, which is a twofold covering of the rotation group. This means that a rotation with an angle of ##2 \pi## maps the spin-state vector not to itself as expected but multiplies it by -1. That's however, a minor detail.
The rotation group is a socalled Lie group, i.e., any group element can be built from infinitesimal transformations, and the corresponding generators are angular-momentum operators ##\hat{J}_k## that obey the commutator relations
$$[\hat{J}_k,\hat{J}_l]=\mathrm{i} \epsilon_{klm} \hat{J}_m.$$
There's one socalled Casimir operator for this Lie algebra, namely the modulus squared of the angular momentum operator ##\hat{\vec{J}}^2=\hat{J}_1^2+\hat{J}_2^2+\hat{J}_3^2##. You can easily check that the three components ##\hat{J}_k## all commute with ##\hat{\vec{J}}^2##, and such an operator is called a Casimir operator of the Lie algebra.
Now it can be shown that all representations of the Lie algebra are uniquely determined by the eigenvalues of ##\hat{\vec{J}}^2##, and the possible eigenvalues are ##J \in \{0,1/2,1,3/2,\ldots \}##. Then the standard basis of each of these representations is chosen as the eigenbasis of ##\hat{J}_z##, and the eigenvalues are ##M_3 \in \{-J,-J+1,\ldots,J-1,J \}##. For ##J=1/2## you get a 2D vector space spanned by the two orthonormal vectors ##|M_3 =\pm 1/2 \rangle##. Each vector in the representation is thus given as a linear combination
$$|\psi \rangle=\psi_{1/2} |M_3=1/2 \rangle + \psi_{-1/2} |M_3=-1/2 \rangle,$$
and the usual convention is to map the vectors to the realization of the 2D vector space to ##\mathbb{C}^2## and to write the components as a column vector (which in this case is called a spinor, because this represents the spin of a spin-1/2 particle):
$$\psi=\begin{pmatrix} \psi_{1/2} \\ \psi_{-1/2} \end{pmatrix}.$$
Now you can also evaluate, using the commutator algebra, how ##\hat{J}_1## and ##\hat{J}_2## act on these basis vectors (you find the derivation in any good textbook on quantum mechanics, e.g., J.J. Sakurai, Modern Quantum Mechanics, Addison Wesley). For convenience one write for the corresponding matrices wrt. to the above basis
$$\hat{J}_k=\frac{1}{2} \hat{\sigma}_k,$$
where the ##\hat{\sigma}_k## are the socalled Pauli matrices. They turn out to be
$$\sigma_1=\begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad \sigma_2=\begin{pmatrix} 0 & -\mathrm{i} \\ \mathrm{i} & 0 \end{pmatrix}, \sigma_3=\begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}.$$
Of course ##\hat{\sigma}_3## is diagonal, because we've chosen the eigenvectors of the 3-component of the spin to determine the basis we want to work with.
Now you can calculate how a rotation acts on the spinors by composing it by infinitesimal transformations, which leads to a matrix exponential. Take the rotation axis (in 3D space!) to be given by the unit vector ##\vec{n}## and the rotation angle ##\varphi##. Then the rotation matrix is given by
$$\hat{R}(\vec{n},\varphi)=\exp \left(-\frac{\mathrm{i}}{2} \varphi \vec{n} \cdot \vec{\sigma} \right ).$$
Using the power series of the exponential function you can show that
$$\hat{R}(\vec{n},\varphi)=\cos (\varphi/2) \hat{1} -\mathrm{i} \vec{n} \cdot \hat{\vec{\sigma}}.$$
As you see for ##\varphi=2\pi## you get ##\hat{R}=-\hat{1}##. To get back to the unit matrix you must set ##\varphi=4 \pi##. That's the above mentioned double cover of the rotation group. You can easily check that the rotation matrices are unitary and that their determinant is 1. Thus we represent the usual rotation grou SO(3) by its covering group SU(2).
I hope this explains a little how the abstract angular-momentum (in this case spin-1/2) eigenvectors in Hilbert space relate to the usual rotations in 3D Euclidean space.
