beginner
14 min read
Exploration vs Exploitation in Reinforcement Learning
Your agent is stuck in a loop. It found one action that earns a small reward, and now it repeats that action forever—never discovering that a much better…
Read tutorialbeginner
Learning From Experience: Exploration and Temporal Difference
Understand how agents collect informative experience, assign credit across time, and update predictions when the environment is unknown.
Tutorials
Articles in Learning From Experience: Exploration and Temporal Difference
Follow the learning path in order or jump directly to the RL concept or experiment you need.
intermediate
10 min read
Monte Carlo vs Temporal-Difference Learning in Reinforcement Learning
Both methods learn from experience. One waits for the story to end before drawing conclusions. The other revises its belief after every sentence. The real…
Read tutorial
intermediate
11 min read
Q-Learning Explained: How Off-Policy Value Learning Finds Better Actions
You can recite the Q-learning update. Maybe you've even coded it in a toy grid world and watched the agent stumble toward a goal. But if someone asked you…
Read tutorial
intermediate
8 min read
SARSA vs Q-Learning: On-Policy and Off-Policy Learning Compared
Two agents start on the same grid. Same states, same actions, same rewards, same exploration schedule. One hugs the edge of a cliff and occasionally falls.…
Read tutorial
intermediate
7 min read
Credit Assignment in Reinforcement Learning: Connecting Late Rewards to Earlier Actions
The agent makes a move. Nothing happens. It makes another move, then another. Twenty steps later, a reward arrives—and the agent has no idea which action…
Read tutorial
beginner
9 min read
A Small Tabular RL Experiment: Watch Exploration Become a Policy
A Q-table makes the invisible visible. Before neural networks, before GPUs, before thousands of parameters, there is a simple grid, a handful of numbers,…
Read tutorial
intermediate
11 min read
Eligibility Traces in Reinforcement Learning: A Bridge Across Time
One-step TD updates are beautifully local: take a step, observe the reward and next state, correct the value of the state you just left. Then you hit the…
Read tutorial
intermediate
8 min read
Q-Learning Overestimation Bias: Why the Max Can Fool the Agent
Your agent's Q-values are climbing. Its actual performance is flatlining. And the gap between what the agent believes and what it achieves keeps widening.
Read tutorial
beginner
9 min read
Contextual Bandits vs Reinforcement Learning: When Actions Don’t Shape Future States
Picture a system that picks which article to show a returning reader. It looks at what it knows about them—their reading history, the time of day, the…
Read tutorial
advanced
11 min read
Intrinsic Motivation in Reinforcement Learning: Exploring When External Rewards Are Sparse
Curiosity is a loan of learning signal. It accelerates exploration when the borrowed interest points toward the goal—and it becomes a debt when the agent…
Read tutorial
intermediate
10 min read
Behavior Cloning and Covariate Shift: Why Demonstrations Fail Off the Training Path
A behavior-cloned agent can look flawless on training data and fall apart the moment it makes one small mistake. That is not bad luck. It is the signature…
Read tutorial
beginner
12 min read
Imitation Learning vs Reinforcement Learning: Copying Behavior or Optimizing Outcomes?
You have a stack of expert demonstrations. Maybe you recorded a human driving a car, or you logged the decisions of a system that already works well. The…
Read tutorial
advanced
10 min read
Inverse Reinforcement Learning: Inferring Goals From Demonstrations
Demonstrations tell you what an expert did. Inverse reinforcement learning asks why—and the answer is never uniquely determined by the behavior itself.
Read tutorial
advanced
11 min read
Exploration Bonuses in RL: Count, Novelty, Prediction Error, and Uncertainty
Epsilon-greedy exploration is randomness without a reason. An exploration bonus is randomness with a budget and a purpose: it changes the reward signal…
Read tutorial
advanced
10 min read
Combining Demonstrations With Reinforcement Learning
Pure reinforcement learning wastes samples rediscovering what a demonstrator already solved. Pure imitation freezes at the demonstrator's ceiling. The real…
Read tutorial