JEE Advanced 2023 · Paper 2, Mathematics question 3

Let $w=\dfrac{6y}{9-y^2}$. For $0<|y|<3$, $w$ has the same sign as $y$.

Case $y>0$. Here $\cot^{-1}\frac1w=\tan^{-1}w$, so the equation is $2\tan^{-1}w=\frac{2\pi}3$, i.e. $w=\sqrt3$:
$$\sqrt3y^2+6y-9\sqrt3=0\implies y=\sqrt3\quad(\text{the other root }-3\sqrt3\text{ is rejected}).$$

Case $y<0$. With $\cot^{-1}$ in $(0,\pi)$, $\cot^{-1}\frac1w=\pi+\tan^{-1}w$. So $2\tan^{-1}w+\pi=\frac{2\pi}3$, giving $w=-\frac1{\sqrt3}$:
$$y^2-6\sqrt3y-9=0\implies y=3\sqrt3-6\approx-0.80\quad(\text{the other root }3\sqrt3+6\text{ is rejected}).$$

Sum. $\sqrt3+3\sqrt3-6=4\sqrt3-6$.