Memory efficient weight loading (#401)

* memory efficient weight loading * remove unused code
rasbt · Oct 14, 2024 · 05b04f2 · 05b04f2
1 parent a20ce1b
commit 05b04f2
Show file tree

Hide file tree

Showing 5 changed files with 1,043 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -118,6 +118,7 @@ Several folders contain optional materials as a bonus for interested readers:
   - [Building a User Interface to Interact With the Pretrained LLM](ch05/06_user_interface)
   - [Converting GPT to Llama](ch05/07_gpt_to_llama)
   - [Llama 3.2 From Scratch](ch05/07_gpt_to_llama/standalone-llama32.ipynb)
+  - [Memory-efficient Model Weight Loading](ch05/08_memory_efficient_weight_loading/memory-efficient-state-dict.ipynb)
 - **Chapter 6:**
   - [Additional experiments finetuning different layers and using larger models](ch06/02_bonus_additional-experiments)
   - [Finetuning different models on 50k IMDB movie review dataset](ch06/03_bonus_imdb-classification)

diff --git a/ch05/08_memory_efficient_weight_loading/README.md b/ch05/08_memory_efficient_weight_loading/README.md
@@ -0,0 +1,5 @@
+# Memory-efficient Model Weight Loading
+
+This folder contains code to illustrate how to load model weights more efficiently
+
+- [memory-efficient-state-dict.ipynb](memory-efficient-state-dict.ipynb): contains code to load model weights via PyTorch's `load_state_dict` method more efficiently