If $\alpha,\beta,\gamma$ (WLOG $\alpha\leq\beta\leq\gamma$) are respectively the angles opposite to $a,b,c$, then $P$ is equal to anyone of $\displaystyle\frac{1}{2}ab\cdot\sin(\gamma),\frac{1}{2}ac\cdot\sin(\beta),\frac{1}{2}bc\cdot\sin( \alpha)$. Hence, $\displaystyle P^3=\frac{1}{8}(abc)^2\sin(\alpha)\sin(\beta)\sin(\gamma)$.
To get an inequality in the right direction, we try to determine the maximum value $m$ of $\sin(\alpha)\sin(\beta)\sin(\gamma)$. Since $\sin$ increases in $[0,\pi/2]$, we have $\sin(\alpha)\leq\sin(\beta)\leq\sin(\gamma)$. So it's not difficult to see that we need to maximize $\alpha$, and that happens for $\alpha=\pi/3$ ($\alpha+\alpha+\alpha\leq\alpha+\beta+\gamma=\pi$). Now, obviously $\alpha=\beta=\gamma=\pi/3$, and hence $\displaystyle m=\sin(\pi/3)\sin(\pi/3)\sin(\pi/3)=(\frac{\sqrt3}{2})^3$.
Finally, $\displaystyle P\leq(\frac{1}{8}(abc)^2m)^{1/3}=\frac{\sqrt3}{4}(abc)^{2/3}$.
Equality occurs precisely for equilateral triangles.
