Skip to content

Commit b6fb74b

Browse files
authored
Update README with training code publication note
Added a note about upcoming training code and pipeline publication for reproducibility.
1 parent cb11d83 commit b6fb74b

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
I've successfully used LEMA to fine-tune "NousResearch/Llama-2-7b-hf" on a custom chat tempalte. While it successfully learned the new vocabulary and special tags, it has not yet mastered the logical structure or grammar of the custom template. You can find the model over [Here](https://huggingface.co/Pomilon/LEMA-llama-2-7b).
66

7+
The training code and pipeline of the model will be polished and published soon for reproducibility.
8+
79
**Virtualize GPU VRAM for LLM Fine-Tuning**
810

911
LEMA is a specialized framework designed to facilitate the fine-tuning of Large Language Models (LLMs) on hardware where model size exceeds available VRAM. By treating model weights as addressable binary segments and implementing a **Triple-Buffer Strategy**, LEMA allows training 7B+ models on GPUs with as little as 16GB VRAM.

0 commit comments

Comments
 (0)