b
Given \(X=Y\)

\(\sqrt{ X ^{2}+ Y ^{2}-2 \times Y \cos \theta}\)

\(= n \sqrt{ X ^{2}+ Y ^{2}+2 \times Y \cos \theta}\)

Square both sides

\(2 X ^{2}(1-\cos \theta)= n ^{2} \cdot 2 X ^{2}(1+\cos \theta)\)

\(1-\cos \theta= n ^{2}+ n ^{2} \cos \theta\)

\(\cos \theta=\frac{1- n ^{2}}{1+ n ^{2}}\)

\(\theta=\cos ^{-1}\left[\frac{ n ^{2}-1}{- n ^{2}-1}\right]\)