\sum produces the summation operator [itex]\sum[/itex]; \Sigma produces a capital sigma [itex]\Sigma[/itex].
You haven't used the correct [itex]z[/itex] component of [itex]\mathbf{f}[/itex]; it should be [itex]z^2[/itex], not [itex]z[/itex] as you have.
As regards the rest of your working, it is simplest to just use [itex]x[/itex] and [itex]y[/itex] as parameters rather than introducing [itex]u[/itex] and [itex]v[/itex]. Then [tex]d\boldsymbol{\sigma} =<br />
\begin{pmatrix} 1 \\ 0 \\ \frac{\partial z}{\partial x} \end{pmatrix} \times<br />
\begin{pmatrix} 0 \\ 1 \\ \frac{\partial z}{\partial y} \end{pmatrix}\,dx\,dy<br />
= \begin{pmatrix} - \frac{\partial z}{\partial x} \\ -\frac{\partial z}{\partial y} \\ 1 \end{pmatrix}\,dx\,dy<br />
= \begin{pmatrix} 3 \\ \tfrac32 \\ 1\end{pmatrix}\,dx\,dy[/tex] and the surface integral then reduces to [tex]
\begin{split}<br />
\iint_\Sigma \mathbf{f} \cdot d\boldsymbol{\sigma} &=<br />
\int_0^1 \int_0^{2(1-x)} 3x^2 + \tfrac32xy + \left(3 - 3x - \tfrac32y\right)^2\,dy \,dx<br />
\\<br />
&= \int_0^1 \int_0^{2(1-x)} \tfrac32 x(2x+y) + \left(3 - \tfrac32(2x+y)\right)^2\,dy\,dx \\<br />
&= \int_0^1 \int_{2x}^2 \tfrac32 xu + 2\left(3 - \tfrac32 u\right)^2\,du\,dx<br />
\end{split}[/tex] using the substitution [itex]u = 2x + y[/itex] in the inner integral (I think this substitution simplifies the algebra significantly, thereby reducing the scope for errors).