In (1/2,0), J_- = 0 and J_+ = (1/2)Pauli, which are 2x2 matrices. In (0,1/2), J_+ = 0 and J_- = (1/2)Pauli, which are again 2x2.
So if you take both to have the 1/2, you would have
J = J_- + J_+ = Pauli - again 2x2 matrices.
I don't see how doing this gives you the canonical 4x4 vector...