PEFT
Parameter-Efficient Fine-Tuning encompasses methods that adapt massive pre-trained language models to new tasks by updating only a tiny fraction of weights.
Think of It Like This
Like customizing a massive, factory-built car by just swapping out the steering wheel and seats, rather than rebuilding the entire engine.
Techniques like LoRA, Adapters, and Prefix Tuning fall under the PEFT umbrella. By keeping the vast majority of the base model's weights frozen, PEFT drastically reduces memory constraints, storage requirements, and training time. It democratizes the customization of billion-parameter models on consumer-grade hardware.