OK, so in other words, you're being asked to find the flux of the field ##\vec F=2y\vec i-z \vec j+x^2\vec k## across the surface S.
First, you should draw the surface and the given 2 planes in the 1st octant so you can better understand the limits and the projection. Projecting the surface S onto the yz-plane will give you an area as shown in the attached figure.
If you use the method as described by LCKurtz, you'll probably find it easier, since it's apparently the standardized method for solving this kind of problem. However, i was taught to use another method, and since it's stuck in my mind, I'm going to use it here and maybe it'll help you to understand better, coupled with whatever method you might already know/use.
$$\phi (x,y,z)=-8x+y^2
\\\nabla \vec \phi=-8\vec i +2y\vec j
\\\hat n =\frac{-8\vec i +2y\vec j}{\sqrt{64+4y^2}}
\\Flux = \int^6_0 \int^4_0 \frac{-8y-yz}{\sqrt{16+y^2}}\,.dydz
$$Solving this double integral should give you the final answer.