Michele LaurelliBlog

GGUF

GGUF: A file format for distributing models and metadata, used primarily by the llama.cpp ecosystem. It packages architecture, tokenizer and tensors, often quantized, in a runtime-readable bundle. Quantization level affects memory, speed and quality. Examples include Q4_K_M model, Local LLM on CPU, Tokenizer metadata.

In italiano: GGUFLegge IV · Loss, gradiente e ottimizzatori

It packages architecture, tokenizer and tensors, often quantized, in a runtime-readable bundle. Quantization level affects memory, speed and quality.

Examples

  • Q4_K_M model
  • Local LLM on CPU
  • Tokenizer metadata