I'm guessing you're getting confused because you've seen different ways to calculate the work, and you're mixing them up. This is why it's important to understand what the equations mean rather than just blindly plugging stuff in. At the very least, you need to know what the variables in an equation mean.
The work done by a constant force F acting over a displacement d is
W = \vec{F}\cdot\vec{d}
In this problem, for example, if you want to calculate the work done by gravity on the block on the incline, F would be the force of gravity, which has a magnitude of Mg and points straight down, and d would be the vector parallel to the incline, pointing in the direction the block moves and having a magnitude of h.
Method 1
You've probably seen two different ways to calculate the dot product, and you can calculate the work by either method. If you know the magnitudes of the vectors and the angle between them, you can find the work using
W = \vec{F}\cdot\vec{d} = |\vec{F}| |\vec{d}| \cos\phi[/itex]<br />
<br />
where ϕ is the angle between the two vectors when placed tail-to-tail. For the block on the incline, the angle between the two vectors is ϕ=θ+90, so the work is<br />
<br />
W = |\vec{F}| |\vec{d}| \cos\phi = Mg \times h \times \cos(\theta+90) = -Mgh\sin\theta<br />
<br />
<span style="font-size: 15px"><b>Method 2</b><br />
<br />
The other way is convenient when you can easily break the vectors up into components. In this case, you'd have<br />
<br />
W = \vec{F}\cdot\vec{d} = F_x d_x + F_y d_y<br />
<br />
where <b>F</b> = F<sub>x</sub><b>i</b>+F<sub>y</sub><b>j</b> and <b>d</b> = d<sub>x</sub><b>i</b>+d<sub>y</sub><b>j</b>. For the incline on the block, the force of gravity points straight down, so <b>F</b>=-Mg<b>j</b>. As you already noted earlier, the displacement in the x-direction is -h cos θ (it moves in the -x direction) and the displacement in the y-direction is h sin θ, so you'd have <b>d</b>=(-h cos θ)<b>i</b>+(h sin θ)<b>j</b>. You'd then calculate the work as follows<br />
<br />
W = F_x d_x + F_y d_y = 0\times(-h \cos\theta) + (-Mg)\times(h \sin \theta) = -Mgh\sin\theta<br />
<br />
You get the same answer as before, as you should. You should note that the only term that contributes to the answer is the y term. Only the vertical displacement matters because the force is vertical. More generally, you can calculate the work by multiplying the magnitude of the force by the displacement in the direction of the force. In your book or notes, you've probably seen this idea expressed as<br />
<br />
W = F d \cos \phi = F (d \cos \phi) = F d_\parallel<br />
<br />
<span style="font-size: 15px"><b>Method 3</b><br />
<br />
In inclined plane problems, sometimes you want to use rotated axes, where the x'-axis runs parallel to the incline and the y'-axis is perpendicular to the incline. Let's assume the +x'-direction points down the incline and the +y'-direction points in the same direction as the normal force. In that case, you'd have <b>F</b>=(Mg sin θ)<b>i</b>+(-Mg cos θ)<b>j</b> and <b>d</b> = -h<b>i</b>, so the work would be<br />
<br />
W = F_{x&#039;} d_{x&#039;} + F_{y&#039;} d_{y&#039;} = (Mg \sin\theta)\times(-h) + (-Mg\cos\theta)\times 0 = -Mgh\sin\theta<br />
<br />
This time, only the x-term contributes to the work. Only the component of the force that points along the displacement matters. So you can calculate the work by multiplying the distance the block moves by the component of the force along the displacement. You might have seen this idea written as<br />
<br />
W = F d \cos \phi = (F \cos \phi)d = F_\parallel d<br />
<br />
So you see, you have several different ways to solve the problem. You just have to keep the details straight.</span></span>