Alright. The unit disk is
S = \{(x, \ y) | x^2 + y^2 \leq 1 \}
Changing to polar coordinates,
S = \{ (\rho \cos{\theta}, \ \rho \sin{\theta}) | \rho \leq 1, \ 0 \leq \theta \leq 2\pi \}
ie. x = \rho \cos{\theta}, \ y = \rho \sin{\theta}.
Thus
\int \int_S xy\sqrt{x^2 + y^2} dA = \int_0^{2\pi} \int_0^1 \rho \sin{\theta}\rho \cos{\theta} \sqrt{\rho^2 \sin^2{\theta} + \rho^2 \cos^2{\theta}} \ \left|\frac{\partial (x, \ y)}{\partial(\rho, \ \theta)}\right| \ d\rho \ d\theta = \int_0^{2\pi} \int_0^1 \rho^4\cos{\theta}\sin{\theta} \ d\rho \ d\theta
I trust you can work out the rest.
I will note that it is quite easy to evaluate this integral without actually doing any calculations, by appealing to symmetry.