v0.20.0

✨ New Features

• Added HiRA: Parameter-Efficient Hadamard High-Rank Adaptation for Large Language Models.
• Added GLoRA: Generalized LoRA for Parameter-Efficient Fine-Tuning, a flexible PEFT method extending LoRA with configurable weight, activation, and bias adaptation.
• Added BEFT: Bias-Efficient Fine-Tuning of Language Models, targeting bias terms for efficient fine-tuning.
• Added MonteCLoRA, a LoRA variant that treats low-rank parameters as a distribution for more robust training.
• Added VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections, targeting activation memory for memory savings.
• Added Uni-LoRA: One Vector is All You Need, using a single global projection for very low parameter counts.
• Added FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence, reconstructing selected weights with shared rotational subspaces.
• Added MiCA: Learns More Knowledge Than LoRA and Full Fine-Tuning, initializing B from the SVD of the base weight.
• Added DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models, splitting weight updates into two learned low-rank parts.
• Added KappaTuneSelector for automatic identification of optimal modules to target based on condition number.
• Added generic quantization support to BOFT, MiSS, SHiRA, and VeRA.
• Multiple adapters now supported with `target_parameters`.
• AdaLoRA can now target `torch.nn.Conv2d` layers via `SVDConv2d`.
• Added an image generation benchmark to the method comparison suite.