JEE Advanced 2020 · Paper 2, Mathematics question 6

Put $x=\frac\pi2+h$.

Numerator.
$$\sin3x+\sin x=2\sin2x\cos x=2(-\sin2h)(-\sin h)\approx4h^2,$$
so the numerator is about $16\sqrt2\,h^2$.

Denominator. Using $2\sin2x\sin\frac{3x}2=\cos\frac x2-\cos\frac{7x}2$, it becomes
$$\cos\tfrac x2-\cos\tfrac{7x}2+\cos\tfrac{5x}2-\cos\tfrac{3x}2-\sqrt2-\sqrt2\cos2x.$$
Expand each term to second order about $\frac\pi2$:

• the constant terms cancel;
• the $h$ terms cancel;
• the $h^2$ coefficient is
  $$\frac{\sqrt2}2\cdot\frac{-1+49+25-9}{8}-2\sqrt2=4\sqrt2-2\sqrt2=2\sqrt2.$$

Limit.
$$\frac{16\sqrt2\,h^2}{2\sqrt2\,h^2}=8.$$
Answer: 8