a
$D = \left| \begin{array}{l}
1\,\,\,1\,\,\,\,1\\
1\,\,\,a\,\,\,1\\
1\,\,\,b\,\,\,1
\end{array} \right|\, = 0$
$ \Rightarrow 1\left[ {a - b} \right]\, - 1\left[ {1 - a} \right] + 1\left[ {b - {a^2}} \right] = 0 \Rightarrow {\left( {a - 1} \right)^2} = 0$
$ \Rightarrow a = 1$
For $ \Rightarrow a = 1$, First two equation are identical ie. $x+y+z=1$
To have no solution with $x+by+z=0$
$b=1$
So $b = \left\{ 1 \right\} \Rightarrow $ It is singleton set.