Yeah, but actually in the meantime I've been looking at this question a bit more closely. It seems the gamma matrices can be used to build representations of the spin(N) algebra, and this is what dirac spinors transform under. But as you say, The Dirac spinor comprises of two Weyl spinors (which might get mixed up by a mass term), each of which transforms in the (0,1/2) and (1/2,0) reps of the Lorentz group. Spin(N) is the double cover of SO(N), so it does 2 copies of the Lorentz transformation basically. And because spinors can be complex, a two component spinor has 4 dofs, which is the correct amount for it to be acted on by an SO(4). But now we have the question - how can we call the theory Lorentz invariant if the things which it is built from, dirac spinors, don't transform under the lorentz group but instead its double cover? I think the answer is that there is nothing stopping us from separating out the left and right handed 2 spinors even when there is a mass term - such a theory now would be invariant under Lorentz since its constructed from 2 spinors. But we run into problems with gauge invraiance (which eventually leads to higgs mechanism), and also for many purposes I think its easier to speak about 4 components spinors and say they transform under Lorentz when what you strictly mean is that the 2 spinors it is comprised of each transforms under different copies of the Lorentz group.