Michele LaurelliBlog

Semantic caching

Semantic caching: A cache that retrieves answers or results based on semantic request similarity rather than exact equality alone. Embeddings and thresholds determine when two requests are equivalent. An overly permissive threshold can return wrong answers, so invalidation and context must be part of the key.

In italiano: Semantic CachingLegge IV · Loss, gradiente e ottimizzatori

Embeddings and thresholds determine when two requests are equivalent. An overly permissive threshold can return wrong answers, so invalidation and context must be part of the key.

Examples

  • Rephrasings of the same question
  • RAG response caching
  • Reducing repeated calls