Skip to content
AI360Xpert
Core ML
Visual explainer

Reinforcement Learning Foundations

How an agent learns from trial and error using delayed feedback and no explicit answer key.

In reinforcement learning, the agent does not receive explicit answers, only delayed numerical feedback after acting.
In reinforcement learning, the agent does not receive explicit answers, only delayed numerical feedback after acting.

Unlike supervised learning, there is no answer key. The agent learns entirely by trial and error, relying on delayed feedback to infer which choices were correct.

The Three Primitives

The three foundational concepts: State, Action, and Reward.
The three foundational concepts: State, Action, and Reward.

The framework consists of three main elements: the State (what the agent sees), the Action (what it chooses to do), and the Reward (the numerical signal it receives).

The Feedback Loop

The agent updates the environment with an action, and the environment answers with the next state and reward.
The agent updates the environment with an action, and the environment answers with the next state and reward.

The agent continuously interacts with the environment. It takes an action, the environment transitions to a new state, and the agent is given a reward. This loop is the basis for all learning.

Maximising Return

The goal of the agent is to maximise its total cumulative reward, also known as the return.
The goal of the agent is to maximise its total cumulative reward, also known as the return.

The agent's objective is not just to get immediate rewards, but to maximise the cumulative return over its entire trajectory. Sometimes, it must sacrifice short-term gains for long-term payoffs.

Where It Breaks

Reward hacking occurs when the agent finds a loophole to maximise points without actually solving the intended problem.
Reward hacking occurs when the agent finds a loophole to maximise points without actually solving the intended problem.

If your reward function is flawed, the agent will exploit it. It will find a loophole to mathematically maximise its score without achieving the actual goal you wanted, a failure mode known as reward hacking.

The Quick Version

  • No answer key, just delayed feedback.
  • State, Action, and Reward form the foundation.
  • Learning happens in an endless interaction loop.
  • The goal is always to maximise total return.
  • Flawed rewards inevitably lead to reward hacking.

What to Read Next