Let's first look at the numerator:
$$\left(x+\sin(x)\right)\left(x\cos(x)+\sqrt{x}\right)^{\prime}-\left(x\cos(x)+\sqrt{x}\right)\left(x+\sin(x)\right)^{\prime}$$
Now, let's look at the parts still left to differentiate, beginning with:
$$\left(x\cos(x)+\sqrt{x}\right)^{\prime}$$
For the first term, we will need to use the product rule:
$$\frac{d}{dx}\left(x\cos(x)\right)=x(-\sin(x))+1(\cos(x))=\cos(x)-x\sin(x)$$
And for the second term, we need the power rule:
$$\frac{d}{dx}\left(\sqrt{x}\right)=\frac{d}{dx}\left(x^{\frac{1}{2}}\right)=\frac{1}{2}x^{-\frac{1}{2}}=\frac{1}{2\sqrt{x}}$$
Putting this together, we may write:
$$\left(x\cos(x)+\sqrt{x}\right)^{\prime}=\cos(x)-x\sin(x)+\frac{1}{2\sqrt{x}}$$
The other part still left to differentiate is:
$$\left(x+\sin(x)\right)^{\prime}=\frac{d}{dx}\left(x+\sin(x)\right)=1+\cos(x)$$
And so the numerator then becomes:
$$\left(x+\sin(x)\right)\left(\cos(x)-x\sin(x)+\frac{1}{2\sqrt{x}}\right)-\left(x\cos(x)+\sqrt{x}\right)\left(1+\cos(x)\right)$$
You could multiply everything out and see if you can combine terms and use trig. identities to simplify.
For the denominator, you have committed an error so commonly made, it has a special name, which is "The Freshman's Dream." You have stated essentially:
$$(a+b)^2=a^2+b^2$$
This is not true in general, what you want, if you are going to expand it, is:
$$(a+b)^2=a^2+2ab+b^2$$