I've tried a lot of things, but I don't get it. (NOT a physics/math student)
I get this:
4cos(θ)^3 - 4cos(θ)
But it should be sin(θ) since I'm applying this formula:
and the solution for 1,1 is Sqrt(3/8π) e^((+/-)iφ) * sin(θ)
I get how the part in italics is derived, but not how the part in bold has been derived from the associated legendre polynomial..
I think you've made an error in your calculation. If we plug in ##\ell = 1## and ##m=1##, we have $$P^{1}_{1}={(-1)^1 \over 2 \cdot 1!} (1-x^2)^{1/2} \cdot {d^2 \over dx^2}(x^2-1)$$
Do you have that much? It should be easy to simplify that, then plug in ## x=\cos{\theta}##.