As matrix $A$ is of order $2,$ let $A=\left[\begin{array}{ll}a & b \\ c & d\end{array}\right]$

Then, $|A|=a d-b c$ and $a d j A=\left[\begin{array}{cc}d & -b \\ -c & a\end{array}\right]$

Now,

$A^{-1}=\frac{1}{|A|} \operatorname{adj} A=\left[\begin{array}{cc}\frac{d}{|A|} & \frac{-b}{|A|} \\ \frac{-c}{|A|} & \frac{a}{|A|}\end{array}\right]$

$\therefore\left|A^{-1}\right|=\left[\begin{array}{ll}\frac{d}{|A|} & \frac{-b}{|A|} \\ \frac{-c}{|A|} & \frac{a}{|A|}\end{array}\right]$

$=\frac{1}{\left|A^{2}\right|}\left|\begin{array}{cc}d & -b \\ -c & a\end{array}\right|$

$=\frac{1}{\left|A^{2}\right|}(a d-b c)$

$=\frac{1}{\left|A^{2}\right|} \cdot|A|=\frac{1}{|A|}$

$\therefore \operatorname{det}\left(A^{-1}\right)=\frac{1}{\operatorname{det}(A)}$

Hence, the correct answer is $D$.