🔧 How do low-rank adaptation of large language models work

Table of Contents



Why study LoRA? The challenge of fine-tuning massive models
Conceptual overview: Full Fine-Tuning
Link back to the Math Equation on Full Fine-tuning
Conceptual Overview: Low-Rank Adaptation (LoRA)
Mathematical Walk-through of Forward Pass with LoRA
Mathematical Walk-through of the Backward Pass
Link Back to the Math Equation...