You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,8 @@
4
4
5
5
I've successfully used LEMA to fine-tune "NousResearch/Llama-2-7b-hf" on a custom chat tempalte. While it successfully learned the new vocabulary and special tags, it has not yet mastered the logical structure or grammar of the custom template. You can find the model over [Here](https://huggingface.co/Pomilon/LEMA-llama-2-7b).
6
6
7
+
The training code and pipeline of the model will be polished and published soon for reproducibility.
8
+
7
9
**Virtualize GPU VRAM for LLM Fine-Tuning**
8
10
9
11
LEMA is a specialized framework designed to facilitate the fine-tuning of Large Language Models (LLMs) on hardware where model size exceeds available VRAM. By treating model weights as addressable binary segments and implementing a **Triple-Buffer Strategy**, LEMA allows training 7B+ models on GPUs with as little as 16GB VRAM.
0 commit comments