I can see what the problem statement seems to be=the line integral around the triangle with vertices at (5,0,0), (0,5,0), and (0,0,5). Instead of trying to integrate across the plane of this triangle, you can also integrate over the surface consisting of the three triangle plane surfaces making up a pyramid. i.e. dydz for the x-component of the curl F at x=0; dxdz for the y-component at y=0; and dxdy for the z component at z=0. I did get zero for an answer when I summed these. I will be glad to check your work to see if you get the same answer I did for each of these. These 2-D integrals are not difficult to evaluate=their limits of integration just take a couple of minutes to compute. ## \\ ## Editing: I also solved it the way you are attempting, but you need to use ## \int \nabla \times \vec{F} \cdot \hat{n} \, dS ##. Your ## \hat{n} ## needs a ##1/\sqrt{3} ## to normalize it which will cancel the ## cos(\gamma) ## factor of ## cos(\gamma)dS=dxdy ## so that ## dS=\frac{dxdy}{cos(\gamma)} ##. (## cos(\gamma)=\hat{n} \cdot \hat{k}=1/\sqrt{3} ##. ). You correctly used ## z=5-x-y ##, but your expression that you integrate will be much simpler if you correctly use ## \nabla \times \vec{F} ## instead of ## \vec{F} ##. And yes, I did get zero for an answer this way as well.
