Using comparative judgments or preferences to infer reward-like objectives and guide reinforcement-learning optimization.
Tagged articles
8 articles in this tag.
More preference labels do not automatically mean a better reward model. The reward model is only as sharp as the comparisons that feed it.
Read tutorial
A reward model can score 95% on the comparisons you trained it on and still be confidently wrong about what people meant. Here is the smallest honest…
Read tutorial
Two reward models, same comparison data, different absolute scores—and both rank every trajectory identically. Neither is broken. The data never pinned…
Read tutorial
Hand-written rewards get hacked. Preference-based reinforcement learning replaces that fragile numeric signal with something that feels safer—human…
Read tutorial
You collected two hundred comparisons. The reward model still ranks trajectories in an order nobody would defend. The instinct is to blame the model. Most…
Read tutorial
If you have worked through policy gradients and deep RL, the language-model post-training landscape looks like a different discipline. RLHF. PPO. DPO.…
Read tutorial
The reward curve climbs steadily. The training loss falls exactly as expected. And the outputs get visibly worse. Human reviewers start rejecting what the…
Read tutorial
A reward model is a compression. Thousands of noisy human comparisons get squeezed into one scalar function, and the compression itself is where meaning…
Read tutorial