GGUF
A specialized file format designed for storing and running large language models efficiently, optimized for fast loading and low memory usage on standard CPUs.
Think of It Like This
Like compressing a massive movie file into a highly optimized format so it can play smoothly on an older laptop without stuttering or crashing the system.
GPT-Generated Unified Format (GGUF) was created by the llama.cpp community to replace the older GGML format. It stores all necessary metadata, tensors, and tokenizers in a single binary file. GGUF is specifically engineered to support mixed-precision quantization, allowing massive open-weights models to run on consumer hardware like MacBooks and standard PCs.