Given $$\displaystyle f(x) = \frac{x^4-x^2}{x^6+2x^3-1}\;,$$ Where $$x>1$$
So We can Simplify $$\displaystyle f(x) = \frac{x^4-x^2}{x^6+2x^3-1} = \frac{x^3\cdot \left(x-\frac{1}{x}\right)}{x^3\cdot \left(x^3-\frac{1}{x^3}\right)+2} = \frac{\left(x-\frac{1}{x}\right)}{\left(x^3-\frac{1}{x^3}\right)+2}$$
Now Let $$u = \left(x-\frac{1}{x}\right)>0\;,\left(x^3-\frac{1}{x^3}\right) = \left(x-\frac{1}{x}\right)^3+3\left(x-\frac{1}{x}\right)\;,x>1.$$
So $$\displaystyle f(u) = \frac{u}{u^3+3u+2}=\frac{u}{u^3+u+u+u+1+1}\leq \frac{u}{6\sqrt[6]{u^3\cdot u \cdot \cdot u \cdot u\cdot 1\cdot 1}} = \frac{1}{6}$$
Using $$\bf{A.M\geq G.M}$$ and above equality hold when $$\displaystyle u = 1\Rightarrow \left(x-\frac{1}{x}\right) = 1\Rightarrow x= \frac{\sqrt{5}+1}{2}$$