Core Architectures (CNN, RNN, Transformer)

Certified Deep Learning Specialist (CDLS) · 50 questions

• A team training a deep feedforward network notices that training loss oscillates wildly and convergence is painfully slow, even after tuning the learning rate. A colleague suggests inserting a normalization layer after each hidden layer's activations. Why would this help?
• In a generative adversarial network, two networks are trained together in competition. What is the specific job of the generator network in this setup?
• A product team wants to build a system that generates long, coherent, human-sounding paragraphs of text from a prompt. Which type of model is best suited to this task?
• During training of a recurrent neural network on long sequences, the loss suddenly spikes to NaN and the weight values become enormous. What phenomenon is most likely responsible?
• In a typical CNN image classifier, the convolutional and pooling layers output a stack of 2D feature maps, but the final classification layers are fully connected and expect a 1D input vector. What layer bridges this gap?
• You need an activation function for the output neuron of a binary classifier so the raw score can be interpreted directly as a probability between 0 and 1. Which activation function fits this requirement?
• What is the core purpose of the self-attention mechanism inside a Transformer block?
• An artificial neuron computes a weighted sum of its inputs, adds a bias term, and passes the result through a nonlinear activation function. What is this process fundamentally intended to model?
• When comparing the internal gating structures of a GRU and an LSTM cell, which statement correctly describes the difference?
• A researcher trains a plain (vanilla) RNN on sequences several hundred time steps long and finds it struggles to learn dependencies between early and late elements in the sequence. What is the primary architectural reason for this limitation?
• A medical imaging team needs a model that labels every individual pixel of an MRI scan as belonging to a specific tissue type, producing a full-resolution segmentation mask rather than a single image-level label. Which architecture is purpose-built for this pixel-wise task?
• In a typical image-classification CNN, the convolutional layers extract low- and mid-level visual features like edges and textures. Which part of the network is responsible for combining those extracted features into the final, high-level representation used to make the classification decision?
• In a basic autoencoder architecture, how do the encoder and decoder halves of the network relate to one another?
• In a convolutional layer, what does the stride parameter actually control?
• A data scientist is building a network to predict house prices, a continuous dollar value with no fixed range. Which activation function should sit on the output layer's single neuron?
• A fraud team has millions of unlabeled, high-dimensional transaction embeddings and wants to flag unusual records without any labeled examples of fraud. Which unsupervised architecture is the classic fit?
• What is the core difference between max pooling and average pooling in a CNN's downsampling layer?
• How does connectivity work between neurons in a fully connected (dense) layer and the layer before it?
• You inherit a trained model and notice its output layer uses a plain linear activation with no squashing at all. What kind of task was this model most likely built for?
• What advantage does a GRU offer compared to a standard (vanilla) RNN when working with longer sequences?
• Mechanically, how is the output of an attention mechanism produced?
• A model performs great on training data but noticeably worse on validation data. A colleague suggests adding a dropout layer. What is dropout actually doing to address this?
• Within an autoencoder, what job does the decoder half of the network perform?
• Which gate inside an LSTM cell is responsible for deciding what information should be discarded from the cell state?
• Since a Transformer processes all tokens in a sequence simultaneously rather than one at a time, why is positional encoding added to the input embeddings?
• A team is building a system that automatically transcribes customer support calls into written text. Which type of model architecture is best suited to this task?
• In a convolutional neural network, what is the main purpose of adding padding around the input feature map before applying a convolution?
• A researcher is choosing between a sigmoid and a tanh activation function for a hidden layer. In which situation would tanh typically be preferred over sigmoid?
• In the context of a convolutional neural network, what does the term 'receptive field' refer to?
• What is the main purpose of a kernel (filter) in a convolutional layer?
• In a feedforward neuron, why is a bias term added alongside the weighted sum of inputs?
• What is the 'bottleneck layer' in an autoencoder?
• In a generative adversarial network (GAN), what is the role of the discriminator network?
• What is a key advantage of a Transformer architecture over a recurrent neural network (RNN) for natural language processing tasks?
• Why do most image-classification CNN architectures end with one or more fully connected layers after the convolutional and pooling layers?
• A team training a GAN to generate images of handwritten digits notices the generator keeps producing only a handful of nearly identical-looking digit images, regardless of the random noise input. What phenomenon is this?
• A company wants to build an unsupervised model that learns to remove background noise from audio recordings, without relying on labeled clean/noisy pairs for every possible noise type. Which architecture is well suited to this?
• What is the main purpose of a variational autoencoder (VAE), as opposed to a standard (vanilla) autoencoder?
• A team has a small dataset of only 800 labeled medical images and wants strong classification performance. They decide to start from a CNN such as VGG or ResNet that was pre-trained on a large general-purpose image dataset. What is the main goal of this transfer learning approach?
• What structurally distinguishes a recurrent neural network from a feedforward neural network?
• What is the primary objective of an encoder-decoder architecture used for sequence-to-sequence (seq2seq) tasks, such as machine translation?
• In a variational autoencoder (VAE), what does the encoder learn to output for a given input, as opposed to a single fixed latent vector?
• What is a key feature of a classic RNN-based encoder-decoder architecture used for machine translation?
• For translating long documents, why is a Transformer generally preferred over an LSTM-based encoder-decoder?
• What is the primary purpose of a residual (skip) connection in a ResNet architecture?
• What is the 'latent space' in an autoencoder?
• In a neural network, what are 'weights'?
• A team wants to build a model that predicts the most likely next word given the words that came before it in a sentence. Which type of architecture is best suited to this task?
• Which activation function is commonly used in deep networks specifically to help mitigate the vanishing gradient problem, compared to sigmoid or tanh?
• What is a key advantage of shared weights (weight sharing) within a convolutional layer of a CNN?