Skip to content

Commit

Permalink
Note about and link to Huggingface
Browse files Browse the repository at this point in the history
  • Loading branch information
dirkgr committed Nov 27, 2024
1 parent 3c6d515 commit 69d1e4e
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ OLMo pretraining follows a two-stage training procedure.
In the first stage, we train on large amounts of mostly web-based data: [OLMo-mix-1124](https://huggingface.co/datasets/allenai/olmo-mix-1124)
In the second stage, we train on a smaller amount of high-quality, targeted data: [Dolmino-mix-1124](https://huggingface.co/datasets/allenai/dolmino-mix-1124)

You can find *all* the checkpoints, at minimum every 1000 training steps, on Huggingface:
* [Huggingface for the 7B variant](https://huggingface.co/allenai/OLMo-2-1124-7B)
* [Huggingface for the 13B variant](https://huggingface.co/allenai/OLMo-2-1124-13B)

#### Stage 1

To get the tokenized training data, look at the paths in the training configs.
Expand Down

0 comments on commit 69d1e4e

Please sign in to comment.