QLoRA
Quantized Low-Rank Adaptation is a highly memory-efficient fine-tuning method that trains low-rank adapters over a frozen, 4-bit quantized base model.
Think of It Like This
Like modifying a massive physical encyclopedia using sticky notes, while leaving the actual heavy pages compressed in a tight, unreadable format.
QLoRA utilizes a specialized 4-bit data type (NormalFloat) and double quantization to shrink a massive LLM into consumer-grade GPU memory. During the forward pass, the frozen weights are momentarily dequantized to interact with the trainable LoRA adapters. It democratized open-source AI by allowing 65B parameter models to be fine-tuned on a single 48GB GPU.