RLHF (Reinforcement Learning from Human Feedback)

RLHF (Reinforcement Learning from Human Feedback): Alignment that uses human preferences to guide a model through reinforcement learning. Preferences often train a reward model, which supplies the signal for policy optimization. Judgment quality, sampling and safeguards against reward hacking shape the outcome.

In italiano: RLHF (Reinforcement Learning from Human Feedback)Legge III · Come una rete impara dai dati

Preferences often train a reward model, which supplies the signal for policy optimization. Judgment quality, sampling and safeguards against reward hacking shape the outcome.

Examples

Comparing two responses
PPO with a learned reward
Penalties for unsafe responses

Fonti

Training language models to follow instructions with human feedback