Llama 3

[awaelchli]

Adds the Llama3 config for the HF models.

Example prompts:

litgpt chat --checkpoint_dir checkpoints/meta-llama/Meta-Llama-3-8B-Instruct

>> Prompt: Tell me a joke.
>> Reply: Here's one:

Why couldn't the bicycle stand up by itself?

(Wait for it...)

Because it was two-tired!

>> Prompt: What is an interesting fact about Paris?
>> Reply: Paris, the City of Light! Here's an interesting fact: Paris has a secret underground tunnel system known as the "Catacombes de Paris" that stretches for over 150 miles (240 kilometers). The tunnels were created in the 18th century by transporting the remains of millions of Parisians from overcrowded cemeteries to the tunnels, where they were stacked in a macabre yet fascinating display. Today, parts of the Catacombes are open to the public for guided tours. Who knew that beneath the City of Romance and Revolution, there was a whole world of hidden history and intrigue?

>> Prompt: What is the square root of i^3, where i is the imaginary unit.
>> Reply: A nice question!

The imaginary unit i is defined as the square root of -1, i.e., i^2 = -1.

Now, let's calculate i^3:

i^3 = i^2 × i = (-1) × i = -i

So, the square root of i^3 is:

√(-i) = ±√(i) × √(-1)
= ±i^(1/2) × i^(1/2)
= ±i

A follow-up PR can add the finetuning configs.

[lantiga]

Looks great! Cannwe start with the llama 2 config for the config_hub for the time being?

[awaelchli]

@lantiga I'll add a finetuning config today

[IvanYashchuk]

What do brackets {} refer to?

[awaelchli]

They are there for string formatting in L883. It gets replaced either with the empty string or with "-Instruct".

[rasbt]

Nice work @awaelchli !!!