What structurally distinguishes a recurrent neural network from a feedforward neural network?
Select an answer to reveal the explanation.
Short Explanation
The whole identity of a recurrent network comes down to one structural trick: cyclic connections. Instead of information flowing strictly forward like in a feedforward net, an RNN loops its hidden state back into itself at the next time step, so information from earlier in a sequence can influence what happens later. That memory-carrying loop is the defining structural difference, so it's the right answer. Batch processing is a training/implementation detail, not an architectural restriction unique to RNNs — they're commonly trained in batches just like any other network. RNN hidden units absolutely use activation functions, typically tanh or sigmoid inside gated units like LSTMs, so that's flat-out wrong. And RNNs are frequently combined with convolutional layers, for example in video processing pipelines that use CNNs for spatial features and RNNs for temporal modeling, so that claim doesn't hold either.
Full Explanation
The key structural distinction between a recurrent neural network (RNN) and a standard feedforward neural network lies in the presence of cyclic (recurrent) connections. In a feedforward network, data flows in a single direction, from the input layer through the hidden layers to the output, with no connections that loop back to a previous point in the computation graph; each input is processed independently of any others. An RNN, by contrast, includes connections that feed a hidden state back into the network at the next time step, alongside the next element of an input sequence. This recurrence allows the network to maintain and update an internal 'memory' as it processes a sequence, enabling it to model dependencies between elements that are separated in time, such as words earlier in a sentence influencing the interpretation of words later on, or earlier time steps in a signal influencing later predictions.
The first distractor, that RNNs can only process a single input at a time and never a batch, confuses the sequential nature of processing time steps within a single sequence with the separate question of batching multiple independent sequences together; RNNs are routinely trained and run in batches, just like feedforward networks, with the batch dimension separate from the time-step dimension.
The second distractor, that RNN hidden units never use an activation function, is factually incorrect; RNNs (and their LSTM/GRU variants) apply activation functions such as tanh or sigmoid within their hidden-state and gating computations, which is essential to introducing the non-linearity needed for the network to learn complex temporal patterns.
The third distractor, that RNNs cannot be combined with convolutional layers, is also incorrect; hybrid CNN-RNN architectures are common, for example, using a CNN to extract spatial features from each video frame and feeding the resulting feature sequence into an RNN to model how those features change temporally across frames.
Memory aid: feedforward equals a one-way street with no loops and no memory of the past; recurrent equals a network with a loop back to itself, carrying a running hidden state forward so it can remember and use information across a sequence.