Generative & Reinforcement Learning

Certified Deep Learning Specialist (CDLS) · 50 questions

• A team is training a sequence model on long text passages and finds that a standard RNN struggles to retain information from the beginning of the sequence by the time it reaches the end. Which architectural change most directly addresses this problem?
• An image classifier needs to correctly identify a cat whether it appears in the top-left corner of a photo or the bottom-right. Which architecture is inherently best suited to handle this kind of variation in feature location?
• What is the primary objective of the policy-gradient method in reinforcement learning?
• In a reinforcement learning system, what role does the policy play?
• A reinforcement learning engineer describes their algorithm as 'off-policy.' What does that mean?
• What is the main role of the environment in a reinforcement learning setup?
• A team training a GAN on a dataset of diverse handwritten digits notices that the generator has started producing only a handful of nearly identical-looking digit images, no matter what noise vector is fed in. What is this phenomenon called?
• What is the main purpose of a variational autoencoder (VAE)?
• What is the main strength of reinforcement learning compared with supervised learning?
• In the standard reinforcement learning framework, who or what is the 'agent'?
• In a standard GAN, what does the generator's loss typically represent during training?
• What role does multi-head attention play in a Transformer architecture?
• In an actor-critic reinforcement learning algorithm, what is the goal of the actor component?
• For an image classification task, why is max pooling often preferred over average pooling?
• A data science team has a large collection of unlabeled images and wants a deep learning model to help group similar images together. Which type of model is commonly used for this?
• What is the main role of the convolutional layer in a CNN?
• What is a key advantage of the Transformer architecture over an RNN for natural language processing tasks?
• A GAN trained to generate faces starts producing only a small set of nearly identical-looking faces regardless of the random input it's given. What GAN-specific problem does this describe?
• What is the main goal of transfer learning in deep learning?
• In Q-learning, what does the Q-value Q(s, a) represent?
• A team wants to compress a large set of unlabeled images into compact representations and later reconstruct them, without using any labeled data. Which architecture is the standard choice?
• How does reinforcement learning fundamentally differ from unsupervised learning?
• Which of the following is a common real-world application of reinforcement learning?
• In a typical CNN architecture, what role does the fully connected layer play after the convolutional and pooling layers?
• What is the main role of the discriminator in a Generative Adversarial Network (GAN)?
• A multi-class image classifier's final layer produces raw, unbounded scores called logits. What does applying a softmax function to these logits accomplish?
• An engineer describes an RL agent that learns action-value estimates from experience gathered under an exploratory behavior policy, while the estimates themselves converge toward the optimal greedy policy. Which classic algorithm best matches this description?
• In reinforcement learning, what does it mean to use a 'policy gradient' method to train an agent?
• A team is deciding which of their four active AI projects best exemplifies a reinforcement-learning problem. Which project is it?
• A researcher stacks 150 convolutional layers and finds training accuracy actually gets worse than with a shallower 50-layer version. A colleague suggests adding skip connections, as in a ResNet architecture. Why would this help?
• When configuring a convolutional layer, an engineer sets the padding parameter to 'same' instead of 'valid'. What does this padding parameter actually control?
• A quant team wants to forecast next-day closing prices using several years of daily historical price and volume data. Which type of architecture is best suited to this time-series prediction task?
• In a reinforcement learning setup for a warehouse robot that picks and moves boxes, what role does the reward function serve?
• What fundamental relationship does the Bellman equation express in reinforcement learning?
• A team has trained a GAN to generate synthetic faces and wants to judge how realistic the output images are. Since there's no simple accuracy metric like in classification, what is a common, practical way to evaluate GAN output quality?
• For an image classification task, an engineer compares a convolutional neural network (CNN) against a plain fully connected feedforward network (FFNN) fed the flattened pixel values. What is the main architectural difference that makes the CNN better suited to images?
• While training a vanilla RNN on long text sequences, an engineer notices the loss occasionally spikes to NaN and weight updates become huge and erratic. What is the standard, straightforward fix for this exploding-gradient problem?
• An autoencoder is trained to reconstruct 256x256 images after passing them through a narrow bottleneck layer of only 32 values. Which component of the autoencoder is responsible for compressing the image down to that low-dimensional representation?
• A robotics team's RL agent first learns to predict how the environment will transition given a state and action, then uses that learned model to simulate and plan ahead before acting in the real world. What kind of RL approach is this?
• In an actor-critic reinforcement learning algorithm, what specific role does the critic play?
• An RL researcher describes a method that updates a state's estimated value immediately after each single step, using the observed reward plus the current estimated value of the next state, rather than waiting for an entire episode to finish. What learning approach is being described?
• A team switches from a simple vanilla RNN to an LSTM for a language modeling task involving paragraphs hundreds of words long. What is the main advantage this switch provides?
• Which characteristic is most central to defining reinforcement learning as a distinct machine learning paradigm?
• A team training a robotic arm to perform complex manipulation tasks first has it practice on simplified, easier versions of the task before gradually introducing full-complexity scenarios. What training strategy is this?
• A newcomer to deep learning asks how a GAN fundamentally differs from an autoencoder, since both are described as 'generative' in some sense. What is the most accurate distinction?
• A marketing team wants to generate entirely new, realistic-looking product photos in various settings that were never actually photographed, to use in ad campaigns. Which architecture is best suited to this image-generation task?
• A neural network with an input layer, three hidden layers, and an output layer has no loops or connections feeding backward between layers during a forward pass. What key characteristic does this describe?
• A convolutional layer applies the exact same filter weights as it slides across every position of an input image, rather than learning a completely separate set of weights for each location. What benefit does this parameter sharing provide?
• Summarized in a single sentence, what is the central idea underlying reinforcement learning as a field?
• A team wants to classify a video clip's overall action (such as 'running' versus 'jumping') by processing its sequence of frames in order, letting information from earlier frames inform interpretation of later ones. Which network type is best suited to this sequential frame classification task?