A square weight matrix needs to be inverted as part of a closed-form solution. Under which condition does that inverse actually exist?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: a square matrix is invertible only when its determinant isn't zero. Think of the determinant as a scaling factor for the space the matrix transforms — if that factor is zero, the transformation squashes everything down into a lower dimension, and you can't undo that squashing. No undo means no inverse. That's why a zero determinant, the first option, is exactly the condition that breaks invertibility rather than allowing it. Symmetry is a nice structural property, but plenty of symmetric matrices are singular, so it doesn't guarantee anything here. And positive entries? Totally irrelevant — you can build a singular matrix out of all positive numbers just as easily as a well-behaved one. Determinant non-zero is the one true test.
Full Explanation
A square matrix A is invertible (also called non-singular) if and only if its determinant is non-zero, det(A) ≠ 0. This is because the determinant represents the scaling factor of the linear transformation that A represents; when det(A) = 0, the transformation collapses the space into a lower dimension (rank deficiency), meaning information is irreversibly lost and no matrix can map the output back to a unique input. Equivalent conditions include: A has full rank, the columns (and rows) of A are linearly independent, and the only solution to Ax = 0 is x = 0.
The first distractor states the opposite of the truth — a zero determinant is precisely the condition under which a matrix is singular and cannot be inverted, so this option describes non-invertibility, not invertibility.
The symmetry distractor is incorrect because symmetry (A = A^T) is an unrelated structural property. Many symmetric matrices are perfectly invertible, but symmetry alone offers no such guarantee: a matrix filled entirely with the same repeated value, for instance, can be symmetric yet singular (determinant zero) because its rows are linearly dependent.
The positive-entries distractor fails for the same kind of reason: the sign of the matrix's entries has no bearing on whether the determinant vanishes. A matrix of all positive numbers can easily have linearly dependent rows and thus a zero determinant.
In deep learning contexts, this matters directly: computing a closed-form solution such as ordinary least squares requires inverting (XᵀX), and if the design matrix has redundant or perfectly correlated features, (XᵀX) becomes singular, forcing practitioners toward regularization (ridge regression) or a pseudoinverse instead. A useful memory aid: 'zero determinant, zero chance of inverting.'