15 lines
791 B
Plaintext
15 lines
791 B
Plaintext
The peft module contains state-of-the-art Parameter-Efficient Fine-Tuning
|
|
(PEFT) methods.
|
|
|
|
Fine-tuning large pretrained models is often prohibitively costly due to their
|
|
scale. Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient
|
|
adaptation of large pretrained models to various downstream applications by only
|
|
fine-tuning a small number of (extra) model parameters instead of all the
|
|
model's parameters. This significantly decreases the computational and storage
|
|
costs. Recent state-of-the-art PEFT techniques achieve performance comparable to
|
|
fully fine-tuned models.
|
|
|
|
PEFT is integrated with Transformers for easy model training and inference,
|
|
Diffusers for conveniently managing different adapters, and Accelerate for
|
|
distributed training and inference for really big models.
|