A team wants to classify a video clip's overall action (such as 'running' versus 'jumping') by processing its sequence of frames in order, letting information from earlier frames inform interpretation of later ones. Which network type is best suited to this sequential frame classification task?
Select an answer to reveal the explanation.
Short Explanation
Since the whole point here is that earlier frames should inform how you interpret later ones, order and memory matter, and that's exactly what RNNs and LSTMs are built for. Feed the frames in one at a time (often after a CNN has already extracted per-frame features), and the recurrent network carries a hidden state forward, accumulating context across the clip so it can recognize a motion pattern like 'running' versus 'jumping' that only makes sense across multiple frames, not from any single frame alone. A fully connected layer applied independently to each frame throws away that temporal context entirely, treating every frame like an unrelated, isolated snapshot. K-means clustering has no supervised classification objective and no sense of sequence either, it's just grouping similar pixels or vectors. And a standalone autoencoder trained only to reconstruct frames is focused on compression and reconstruction, not on classifying an action across a temporal sequence.
Full Explanation
For tasks that require classifying an action or event that unfolds across a sequence of video frames, recurrent architectures such as RNNs and, more commonly in practice, LSTMs (or GRUs) are the natural fit because they are explicitly designed to process sequential data while maintaining and updating an internal hidden state that carries information forward from earlier elements in the sequence to influence the interpretation of later ones. In a typical video-action-classification pipeline, each frame is often first passed through a convolutional neural network to extract a compact feature vector capturing that frame's visual content, and this sequence of per-frame feature vectors is then fed into a recurrent network, one time step per frame, allowing the network to build up a representation of the motion and temporal pattern across the whole clip, which is essential for distinguishing actions like running versus jumping that are defined by how frames change over time rather than by any single static frame. This combined CNN-plus-RNN/LSTM approach (or, in more modern systems, 3D convolutions or video transformers) is a standard strategy precisely because a single frame in isolation often cannot disambiguate the ongoing action.
The distractor describing a fully connected layer applied independently to each frame with no memory is incorrect because it discards exactly the temporal context the task requires; without any mechanism to relate one frame's representation to the next, the network cannot learn patterns that are defined by motion or change over time, and would have to attempt to classify actions from single, isolated frames, which is often ambiguous or impossible. The k-means clustering distractor is incorrect because clustering is an unsupervised technique for grouping similar data points based on a distance metric, with no notion of temporal order, sequence modeling, or a supervised classification objective mapping sequences to action labels; it is not designed to solve a supervised sequence classification problem at all. The standalone-autoencoder distractor is incorrect because an autoencoder trained only to reconstruct individual frames is focused on learning a compressed representation for reconstruction of that single frame, with no built-in mechanism for relating information across frames over time or predicting an action label; it addresses a fundamentally different objective (unsupervised representation learning and reconstruction) rather than supervised temporal action classification.
A helpful memory aid: whenever a task depends on 'what happened before this moment' to correctly interpret 'what's happening now,' whether it's language, audio, or a sequence of video frames, recurrent architectures like RNNs and LSTMs (or their modern attention-based successors) are the tools built specifically to carry that context forward through time.