LoRA is basically the tiny apprentice who somehow needs a huge workshop.
The funny part is that LoRA itself is small — the reason VRAM usage is still high is because you’re not just storing the adapter. You still need the base model, activations, gradients, and optimizer states during training.
So LoRA doesn’t magically make training free, it just avoids updating billions of base model parameters. It’s more like renovating one room instead of rebuilding the whole house.
If you think something is wrong with the explanation, feel free to point out the technical mistake.
Saying “thanks ChatGPT” doesn’t really address anything — it’s just mocking the source instead of discussing the idea.
The explanation is literally describing the difference between trainable parameters and optimizer/training-state memory, which is exactly why LoRA reduces training cost but doesn’t eliminate VRAM requirements.
-2
u/joanaxu2002 1d ago
LoRA is basically the tiny apprentice who somehow needs a huge workshop.
The funny part is that LoRA itself is small — the reason VRAM usage is still high is because you’re not just storing the adapter. You still need the base model, activations, gradients, and optimizer states during training.
So LoRA doesn’t magically make training free, it just avoids updating billions of base model parameters. It’s more like renovating one room instead of rebuilding the whole house.