BM25
BM25: A lexical ranking function that scores documents using term frequency, rarity and document length. Saturation limits the benefit of repeating a term, while IDF rewards rare terms. Parameters k1 and b control term frequency and length normalization respectively. Examples include Full-text search, Baseline for a RAG pipeline, Ranking with Elasticsearch.
In italiano: BM25Legge I · Embedding, feature e spazi vettorialiSaturation limits the benefit of repeating a term, while IDF rewards rare terms. Parameters k1 and b control term frequency and length normalization respectively.
Formula
score(D,Q) = Σᵢ IDF(qᵢ) · [f(qᵢ,D)(k₁+1)] / [f(qᵢ,D)+k₁(1−b+b|D|/avgdl)]
Examples
- Full-text search
- Baseline for a RAG pipeline
- Ranking with Elasticsearch