$ab(a^2+b^2)=ab(9-2ab)=9ab-2(ab)^2$, let $ab=k$, maximum of $9k-2k^2$ is when $k=\frac{9}{4}$. Rest calculation is for you.