LLM-as-a-judge
LLM-as-a-judge: Using a model to evaluate other models’ responses according to an explicit rubric. It scales qualitative assessment but introduces preferences for style, position and longer responses. Calibration against human judgments and cross-checks estimate judge reliability. Examples include Completeness and faithfulness rubric, Pairwise comparison, Grounding evaluation.
In italiano: LLM-as-a-JudgeLegge V · Overfitting, bias e collasso dei modelliIt scales qualitative assessment but introduces preferences for style, position and longer responses. Calibration against human judgments and cross-checks estimate judge reliability.
Examples
- Completeness and faithfulness rubric
- Pairwise comparison
- Grounding evaluation