You need to find the maximum of a function. The maximum of a function is given when f'(x) = 0. So taking Planck's radiation formula for energy density per unit wavelength;
Once you have found the derivative all that remains is to solve the equation. As far as I know there exists no analytical solution to the equation and it must be solved numerically. However, if I have time I may venture into the maths forums and inquire as to whether an analytical solution exists.
HINT: Solve numerically for [itex]\lambda_{max}T[/itex] first.