Automated formatting (#1180)

README.md

@@ -401,4 +401,3 @@ If you use LitGPT in your research, please cite the following work:
 ## License
 
 LitGPT is released under the [Apache 2.0](https://github.com/Lightning-AI/litgpt/blob/main/LICENSE) license.
-

config_hub/finetune/README.md

@@ -1,6 +1,6 @@
 ## Config files
 
-The table below lists the performances you can expect from the provided config files. Note that you can achieve lower memory consumption by lowering the micro batch size as needed. In addition, you can lower the rank (`lora_r`) in the LoRA configuration files and disable LoRA for certain layers (for example, setting `lora_projection` and other LoRA layer-specific parameters to `false`). 
+The table below lists the performances you can expect from the provided config files. Note that you can achieve lower memory consumption by lowering the micro batch size as needed. In addition, you can lower the rank (`lora_r`) in the LoRA configuration files and disable LoRA for certain layers (for example, setting `lora_projection` and other LoRA layer-specific parameters to `false`).
 For more information, see the [Dealing with out-of-memory (OOM) errors](../../tutorials/oom.md) on lowering the memory requirements.
 
  

extensions/thunder/README.md

@@ -40,7 +40,7 @@ print(forward_trace)
 @torch.no_grad()
 @no_autocast()
 def augmented_forward_fn(*args):
-  # args: "Collection" 
+  # args: "Collection"
   t0, \
   t1, \
   t2, \
@@ -245,8 +245,8 @@ print(backward_trace)
 @torch.no_grad()
 @no_autocast()
 def backward_fn(saved_for_backward, cotangents):
-  # saved_for_backward: "Collection" 
-  # cotangents: "Collection" 
+  # saved_for_backward: "Collection"
+  # cotangents: "Collection"
   C0, \
   C1, \
   = saved_for_backward
@@ -528,7 +528,7 @@ We provide ready-to-use Fabric strategies that integrate Thunder DDP|FSDP. Under
 
 ```python
 model = thunder.distributed.ddp(model)
-# or 
+# or
 # model = thunder.distributed.fsdp(model)
 
 model = thunder.jit(model)

extensions/thunder/unsloth/pretrain.py

@@ -439,11 +439,10 @@ def validate_args(train: TrainArgs, eval: EvalArgs, initial_checkpoint_dir, resu
 
 def jit(fn: Callable) -> Any:
     import thunder
+    from executor import unsloth_ex
     from thunder.executors.sdpaex import sdpa_ex
     from thunder.executors.torch_compile import torch_compile_executor
 
-    from executor import unsloth_ex
-
     return thunder.jit(
         fn, executors=[sdpa_ex, unsloth_ex, torch_compile_executor, thunder.nvfuser_executor, thunder.pytorch_executor]
     )

litgpt/data/json_data.py

@@ -18,8 +18,8 @@ class JSON(DataModule):
     """Loads JSON or JSONL data for supervised finetuning."""
 
     json_path: Path
-    """A path to a JSON file or a directory with `train.json` and `val.json` containing the data. 
-    The file(s) should contain a list of samples (dicts). Each dict must have the keys 'instruction' and 'output', 
+    """A path to a JSON file or a directory with `train.json` and `val.json` containing the data.
+    The file(s) should contain a list of samples (dicts). Each dict must have the keys 'instruction' and 'output',
     and can optionally have a key 'input' (see Alpaca)."""
     mask_prompt: bool = False
     """Whether to mask the prompt section from the label (with ``ignore_index``)."""

tests/data/fixtures/alpaca.json

@@ -114,4 +114,4 @@
         "input": "Recording about the history of Waterloo",
         "output": "The Battle of Waterloo, fought on June 18, 1815, was the last major battle of the Napoleonic Wars. It marked the end of Napoleon Bonaparte's rule. The battle was fought between Napoleon Bonaparte's French forces and an allied force of British, Dutch, Belgian and Prussian troops, led by the Duke of Wellington. It was fought near the town of Waterloo in the former Austrian Netherlands, now present-day Belgium. The battle was a defeat for Napoleon's forces and ended Napoleon's rule over France. Napoleon himself was exiled to St. Helena, where he would die in 1821. The battle marked the end of French domination of Europe and the beginning of a period of peace in the region. It was also the last major battle of the Napoleonic Wars and is remembered as one of the most important battles in history. The victory of the allied forces at Waterloo helped to bring an end to French imperial ambitions in Europe. It also established the British as one of the most powerful nations in Europe. The battle was fought with a combination of infantry, cavalry, and artillery tactics, and showed the beginning of the changing nature of warfare. Although the French forces greatly outnumbered the allies, their strategic and tactical mistake led to the loss of the battle. This defeat signaled the end of French imperial power in Europe. The legacy of Waterloo still stands today, and it cemented the British Empire's position for the next one hundred years."
     }
-]
+]

tests/data/fixtures/longform_train.json

@@ -101,4 +101,4 @@
     "source": "Wikipedia",
     "subset": "chatbot"
   }
-]
+]

tests/data/fixtures/longform_val.json

@@ -53,4 +53,4 @@
     "source": "Natural Instructions",
     "subset": "task1658 billsum summarization"
   }
-]
+]

tests/test_thunder_fsdp.py

@@ -6,11 +6,10 @@
 
 import pytest
 import torch
+from conftest import RunIf
 from lightning.fabric import Fabric
 from lightning.fabric.utilities.imports import _TORCH_GREATER_EQUAL_2_3
 
-from conftest import RunIf
-
 # support running without installing as a package
 wd = Path(__file__).parent.parent.resolve()
 sys.path.append(str(wd))

tests/test_unsloth_executor.py

@@ -3,8 +3,8 @@
 from conftest import RunIf
 
 from litgpt import GPT, Config
-from litgpt.utils import chunked_cross_entropy
 from litgpt.model import apply_rope, build_rope_cache
+from litgpt.utils import chunked_cross_entropy
 
 
 @RunIf(min_cuda_gpus=1, thunder=True)
@@ -85,9 +85,9 @@ def test_unsloth_swiglu():
     import thunder
     from thunder.core.transforms import grad
 
-    from extensions.thunder.unsloth.executor import unsloth_ex, ThunderLLaMAMLP
-    from litgpt.model import LLaMAMLP
+    from extensions.thunder.unsloth.executor import ThunderLLaMAMLP, unsloth_ex
     from litgpt import Config
+    from litgpt.model import LLaMAMLP
 
     config = Config.from_name("Llama-2-7b-hf")
     with torch.device("cuda"):

tutorials/convert_hf_checkpoint.md

@@ -26,13 +26,13 @@ checkpoints/
 To disable the automatic conversion, which is useful for development and debugging purposes, you can run the `litgpt/scripts/download.py` with the `--convert_checkpoint false` flag. This will only download the checkpoint files but do not convert them for use in LitGPT:
 
 ```bash
-rm -rf checkpoints/EleutherAI/pythia-14m 
+rm -rf checkpoints/EleutherAI/pythia-14m
 
 litgpt download \
   --repo_id EleutherAI/pythia-14m \
   --convert_checkpoint false
-  
-ls checkpoints/EleutherAI/pythia-14m 
+
+ls checkpoints/EleutherAI/pythia-14m
 ```
 
 ```
@@ -52,4 +52,3 @@ The required files `model_config.yaml` and `lit_model.pth` files can then be man
 litgpt convert to_litgpt \
   --checkpoint_dir checkpoints/EleutherAI/pythia-14m
 ```
-

tutorials/convert_lit_models.md

@@ -66,7 +66,7 @@ For convenience, we first specify an environment variable (optional) to avoid co
 export repo_id=TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
 ```
 
-Instead of using TinyLlama, you can replace the `repo_id` target with any other model repository 
+Instead of using TinyLlama, you can replace the `repo_id` target with any other model repository
 specifier that is currently supported by LitGPT. You can get a list of supported repository specifier
 by running `litgpt/scripts/download.py` without any additional arguments.
 
@@ -147,4 +147,4 @@ lm_eval --model hf \
     --tasks "hellaswag,gsm8k,truthfulqa_mc2,mmlu,winogrande,arc_challenge" \
     --device "cuda:0" \
     --batch_size 4
-```
+```

tutorials/download_model_weights.md

@@ -209,7 +209,7 @@ litgpt chat --checkpoint_dir checkpoints/$repo_id
  
 ## Specific Models
 
-Note that certain models require that you've been granted access to the weights on the Hugging Face Hub. 
+Note that certain models require that you've been granted access to the weights on the Hugging Face Hub.
 
 For example, to get access to the Gemma 2B model, you can do so by following the steps at https://huggingface.co/google/gemma-2b. After access is granted, you can find your HF hub token in https://huggingface.co/settings/tokens.
 
@@ -249,7 +249,7 @@ litgpt download \
  
 ## Converting Checkpoints Manually
 
-For development purposes, for example, when adding or experimenting with new model configurations, it may be beneficial to split the weight download and model conversion into two separate steps. 
+For development purposes, for example, when adding or experimenting with new model configurations, it may be beneficial to split the weight download and model conversion into two separate steps.
 
 You can do this by passing the `--convert_checkpoint false` option to the download script:
 

tutorials/prepare_dataset.md

@@ -80,7 +80,7 @@ For comparison, the Falcon 7B model requires 23.52 GB of memory for the original
 ### Alpaca-GPT4
 
 
-The Alpaca-GPT4 was built by using the prompts of the original Alpaca dataset and generate the responses via GPT 4. The 
+The Alpaca-GPT4 was built by using the prompts of the original Alpaca dataset and generate the responses via GPT 4. The
 dataset consists of 52,000 instructions and responses.
 
 The original [Alpaca-GPT4](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM) dataset can be used as follows:
@@ -131,7 +131,7 @@ litgpt finetune lora \
 
 ### Deita
 
-The Deita dataset (short for Data-Efficient Instruction Tuning for Alignment) is a collection of 9500 prompts and responses, as described in the [What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning](https://arxiv.org/abs/2312.15685) paper. 
+The Deita dataset (short for Data-Efficient Instruction Tuning for Alignment) is a collection of 9500 prompts and responses, as described in the [What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning](https://arxiv.org/abs/2312.15685) paper.
 Using Falcon 7b as an example, we can use the dataset as follows:
 
 ```bash