What is the primary objective of the policy-gradient method in reinforcement learning?
Select an answer to reveal the explanation.
Short Explanation
Policy-gradient methods skip the middleman. Instead of first learning value estimates for every state and action and then deriving a policy from them, you directly parameterize the policy itself — usually as a neural network — and nudge its weights, using gradient ascent, in the direction that increases expected reward. That's the whole idea: directly optimize a policy that maps states to actions. There's no lookup table being built (that's more of a tabular value-based approach and doesn't scale anyway), there's no labeled 'correct action' data (RL doesn't work like supervised learning — you only get rewards, not answer keys), and there's no clustering step involved. You're adjusting policy parameters based on how much reward the actions the policy chose actually earned.
Full Explanation
The policy-gradient family of methods (REINFORCE, actor-critic variants, PPO, TRPO, etc.) works by directly parameterizing the policy — a function, typically a neural network with parameters theta, that maps states to a probability distribution over actions — and then using gradient ascent to adjust those parameters in the direction that increases expected cumulative reward. This is fundamentally different from value-based methods like Q-learning, which first learn value estimates (Q-values) for state-action pairs and derive a policy indirectly (e.g., by acting greedily with respect to those values). Policy-gradient methods compute an estimate of the gradient of the expected return with respect to the policy parameters and update the parameters accordingly, which makes them naturally suited to continuous or high-dimensional action spaces where value-based lookup approaches become impractical. Building a lookup table of exact rewards for every state describes a tabular, model-free value-based approach at best, and is not scalable or accurate for most real problems since it ignores generalization entirely — it also is not what policy gradients do, since policy gradients never build such a table. Labeling training data with the 'correct' action per state describes supervised learning (e.g., behavior cloning), which requires ground-truth labels; reinforcement learning, including policy-gradient methods, instead relies solely on a scalar reward signal from the environment and does not require labeled optimal actions. Clustering states before learning is an unsupervised preprocessing idea unrelated to how policy gradients function; policy-gradient methods learn the state-to-action mapping directly through interaction and reward feedback, not through a separate state-grouping step. In short, the policy-gradient method's defining characteristic is that it optimizes the policy function itself, end to end, via gradients estimated from sampled trajectories and their resulting rewards.