Learning or evaluating policies from fixed logged experience without relying on unrestricted new interaction, including support and extrapolation limits.
Tagged articles
9 articles in this tag.
You train an offline agent on a fixed dataset. The final score looks strong. The learning curves climb. You feel the quiet satisfaction of a result that…
Read tutorial
Imagine training an offline agent and watching its Q-values climb steadily. The critic grows more confident. The policy selects actions with increasing…
Read tutorial
You have a dataset of logged trajectories collected under one policy, and you want to know what a different policy would have earned. The naive…
Read tutorial
You have a log of episodes from one policy, a candidate policy you have never deployed, and a single number that claims to be the candidate's value. The…
Read tutorial
You have a new policy that looks promising. The next step feels obvious: deploy it, watch it interact with the real environment, and measure what happens.…
Read tutorial
A candidate policy that looks better on paper is not the same as a candidate policy that is better. Here is one small, honest rule for telling the…
Read tutorial
Two estimators, one dataset, two different answers. That gap is not a bug in the math — it is the most useful signal you will get about whether your logs…
Read tutorial
An offline RL agent can report a sky-high value for an action it never actually tried. The critic is extrapolating into territory the dataset never…
Read tutorial
A higher estimated value is not proof of improvement. It is a hypothesis about actions the dataset may never have supported.
Read tutorial