intermediate
8 min read
Trace a Linear Counterexample to the Deadly Triad
Everyone repeats that the deadly triad diverges. Almost nobody watches it happen. So let's build a predictor small enough to update by hand, then watch the…
Read tutorialconcept
Function Approximation
Using parameterized representations such as neural networks instead of tables, including generalization and approximation-related behavior.
Tagged articles
Function Approximation articles
15 articles in this tag.
intermediate
9 min read
The Deadly Triad in Reinforcement Learning: Why Learning Can Become Unstable
Your Q-learning agent trains beautifully for ten thousand steps. The loss curve dips, the reward trend climbs, and you start to believe the hard part is…
Read tutorial
intermediate
9 min read
Deep Reinforcement Learning Explained: What Neural Networks Add
When you hear "deep reinforcement learning," it's tempting to picture a neural network that somehow learned to think for itself. The network isn't doing…
Read tutorial
intermediate
8 min read
Build a Small DQN Agent in Python: Replay Buffers and Target Networks
You can explain replay buffers and target networks. You have never watched a loss curve flatline while episode return refuses to move. That gap between…
Read tutorial
advanced
10 min read
Extrapolation Error in Offline RL: Why Unseen Actions Mislead the Agent
Imagine training an offline agent and watching its Q-values climb steadily. The critic grows more confident. The policy selects actions with increasing…
Read tutorial
advanced
10 min read
Bellman Error With Function Approximation: Why Low Training Loss Can Mislead
Your value network's loss curve flattens to something that looks like zero. The run looks healthy. Then you watch the greedy policy walk straight into a…
Read tutorial
advanced
10 min read
Why Monotonic Value Factorization Supports Decentralized Greedy Actions
A team where every agent picks its own best action can still lose. Monotonic value factorization is the constraint that rules out one specific failure —…
Read tutorial
advanced
11 min read
Offline Reinforcement Learning Explained: Learning From a Fixed Dataset
An offline RL agent can report a sky-high value for an action it never actually tried. The critic is extrapolating into territory the dataset never…
Read tutorial
intermediate
9 min read
Parameterized Policies in Reinforcement Learning: From Tables to Networks
A policy network is not a bigger table. That is the mental model to dismantle before you touch policy-gradient code, because it quietly teaches you the…
Read tutorial
intermediate
11 min read
Stabilizing Deep Q-Learning: Replay, Target Networks, and Moving Targets
Tabular Q-learning converges reliably. Swap the table for a neural network, and the same update can oscillate, blow up, or quietly forget everything it…
Read tutorial
intermediate
12 min read
When Does Tabular Q-Learning Converge? Conditions and Limits
Your Q-table went quiet. The values stopped moving, the greedy policy stopped changing, and the learning curve flattened into something that looks like a…
Read tutorial
intermediate
7 min read
Tabular RL vs Function Approximation: What Changes When Tables Become Networks?
Your Q-table worked beautifully on the grid world. Every cell held a number, every update made sense, and you could read the agent's beliefs like an open…
Read tutorial
advanced
12 min read
TD3 Explained: Twin Critics and Delayed Updates for Continuous Control
DDPG doesn't fail because its policy is weak. It fails because its critic quietly starts lying, and the policy trusts every word.
Read tutorial
advanced
11 min read
Value Decomposition in Multi-Agent RL: Learning Team Actions From Local Views
A team shares one reward, but at execution time each agent must pick its own action from only its own observation. Value decomposition answers that…
Read tutorial
intermediate
10 min read
When Deep Reinforcement Learning Is the Wrong Tool
Deep RL is not the destination of reinforcement learning. It is a specialized tool with a steep cost profile, and most learners reach for it before they…
Read tutorial