beginner
13 min read
How to Design a Reinforcement Learning Environment
Most beginners assume the algorithm is the hard part of reinforcement learning. Pick the right one, tune it well, and the agent will learn. Then they spend…
Read tutorialintermediate
Designing and Evaluating RL Experiments
Teach readers to turn algorithms into trustworthy experiments by defining environments, choosing metrics, diagnosing failure, and separating learning progress from lucky episodes.
Tutorials
Articles in Designing and Evaluating RL Experiments
Follow the learning path in order or jump directly to the RL concept or experiment you need.
intermediate
8 min read
How to Evaluate Reinforcement Learning Agents Beyond One Good Episode
Your agent just nailed an episode. It reached the goal, collected a beautiful stream of rewards, and you felt that rush of validation. Then you rerun it,…
Read tutorial
intermediate
10 min read
Why Reinforcement Learning Agents Fail: A Diagnostic Guide
Your agent runs. The loop completes. No exceptions are raised. And yet the reward curve sits there like a flatline—or climbs promisingly for two hundred…
Read tutorial
intermediate
11 min read
Reward Shaping vs Sparse Rewards in Reinforcement Learning
Your agent is doing one of two things: nothing, or the wrong thing. When rewards never arrive, it wanders forever without learning. When rewards arrive too…
Read tutorial
intermediate
10 min read
Baselines and Ablations for Reinforcement Learning Experiments
You changed two things at once. The agent got better. And now you have no idea which change earned the improvement.
Read tutorial
absolute beginner
9 min read
A Reinforcement Learning Learning Path: From Agent Loop to Deep RL
Most people start reinforcement learning the wrong way. They install a deep RL library, copy a DQN or PPO example, watch the agent fail—or worse, behave…
Read tutorial
beginner
9 min read
Learning Rate and Discount Factor in RL: A Small Experiment With Different Behaviors
Two knobs. One update rule. If you are starting out in reinforcement learning, it is tempting to treat the learning rate and discount factor as twin dials…
Read tutorial
intermediate
9 min read
The Deadly Triad in Reinforcement Learning: Why Learning Can Become Unstable
Your Q-learning agent trains beautifully for ten thousand steps. The loss curve dips, the reward trend climbs, and you start to believe the hard part is…
Read tutorial
intermediate
12 min read
Generalization in Reinforcement Learning: Does the Policy Transfer?
Your agent crushes its training level. Every episode, the same flawless path, the same high score, the same satisfying convergence curve. Then you move one…
Read tutorial
intermediate
9 min read
Invalid Actions in Reinforcement Learning: Masks, Penalties, and Hidden Bugs
Your agent keeps walking into walls. It selects skills still on cooldown, tries to buy items it cannot afford, and bumps against the grid edge like a moth…
Read tutorial
advanced
11 min read
Offline Reinforcement Learning Explained: Learning From a Fixed Dataset
An offline RL agent can report a sky-high value for an action it never actually tried. The critic is extrapolating into territory the dataset never…
Read tutorial
advanced
10 min read
Extrapolation Error in Offline RL: Why Unseen Actions Mislead the Agent
Imagine training an offline agent and watching its Q-values climb steadily. The critic grows more confident. The policy selects actions with increasing…
Read tutorial
advanced
12 min read
Sim-to-Real Reinforcement Learning: Why a Policy Breaks Outside the Simulator
A policy that scores perfectly in simulation is not a policy that works in the real world. It is a policy that solved the simulator.
Read tutorial
intermediate
13 min read
Off-Policy Evaluation in Reinforcement Learning: Estimating a Policy From Logged Experience
You have a new policy that looks promising. The next step feels obvious: deploy it, watch it interact with the real environment, and measure what happens.…
Read tutorial
advanced
11 min read
Safe Policy Improvement From Logged Data: Staying Near What the Dataset Supports
A higher estimated value is not proof of improvement. It is a hypothesis about actions the dataset may never have supported.
Read tutorial
intermediate
11 min read
Dataset Support and Coverage in Offline RL: Can the Logs Answer Your Question?
You train an offline agent on a fixed dataset. The final score looks strong. The learning curves climb. You feel the quiet satisfaction of a result that…
Read tutorial
advanced
11 min read
Importance Sampling and Doubly Robust Evaluation in Offline RL
You have a dataset of logged trajectories collected under one policy, and you want to know what a different policy would have earned. The naive…
Read tutorial
advanced
8 min read
Constrained Reinforcement Learning: Optimizing Reward Without Breaking the Rules
A penalty is a price. A constraint is a wall. Most RL safety problems fail because practitioners build walls out of prices and then act surprised when the…
Read tutorial
advanced
11 min read
Safe Exploration in Reinforcement Learning: Action Shields, Rules, and Fallbacks
Your agent proposes an action that crosses a hard safety boundary. You can block it, replace it, or fall back to a known-safe controller. The naive…
Read tutorial
advanced
14 min read
Risk-Sensitive Reinforcement Learning: Choosing Policies Beyond Average Return
Two policies can share the exact same expected return and still feel completely different to the person who has to live with the outcomes. One delivers…
Read tutorial
advanced
11 min read
Evaluating Exploration in RL: Did the Agent Discover Useful Information?
A novelty bonus can make your agent look busy without making it look smart. The intrinsic-reward curve climbs, the state-visitation heatmap fills in, and…
Read tutorial