In the context of the mathematics used by deep learning frameworks, what does taking the tensor product of two tensors do?
Select an answer to reveal the explanation.
Short Explanation
Think of the tensor product like combining two separate LEGO pieces into one bigger, more complex structure — not squishing them together, but building something with more dimensions than either piece had alone. That's the tensor product: it takes two tensors and produces a new one with a higher rank (more axes), formed from every combination of their entries. That's B. Element-wise multiplication (the first option) needs matching shapes and keeps the same rank — that's a totally different operation. Reducing to a scalar sounds like a dot product or a full contraction, not a tensor product. And stacking along an existing axis is just concatenation, which doesn't add a new dimension the way a tensor product does.
Full Explanation
The tensor product (sometimes called the outer product when applied to vectors) takes two tensors and produces a new tensor whose rank is the sum of the ranks of the inputs. Every entry of the first tensor gets multiplied against every entry of the second, and the results are arranged into a new, higher-dimensional array. For example, the outer product of two vectors (rank-1 tensors) produces a matrix (rank-2 tensor); the tensor product of two matrices produces a rank-4 object. This operation is foundational in areas like building joint representations, certain attention mechanisms, and constructing higher-order interaction terms from lower-order features.
The first distractor describes element-wise (Hadamard) multiplication, which requires the two tensors to already share the same shape and produces an output of the same rank as the inputs — it does not increase dimensionality, and it is a fundamentally different operation from the tensor product.
The third distractor describes a full contraction or an inner/dot product, where summing over matching indices collapses the tensors down toward a scalar (for two vectors) or a lower-rank result. This is close to the opposite of what a tensor product does, since the tensor product expands rank rather than collapsing it.
The fourth distractor describes concatenation, which joins tensors along an existing axis and therefore keeps the same total number of axes (rank), only changing the size of that one axis. No new dimension is introduced, which is the key difference from a true tensor product.
Memory aid: dot product contracts and shrinks rank; tensor product expands and grows rank. If an operation's result has more axes than either input, you are looking at a tensor product.