demo example / number of parameter control vs original code

[thegodone]

Excellent, Can you write a demo example on training ? Comparing to the Microsoft code have you check that it provide same number of parameters for the same settings ?

[Jamie-Stirling]

This is a good idea, thank you.

I'll look into this when I get time, hopefully tomorrow.

[thegodone]

Great, maybe you can reused this example on harry potter: https://github.com/DonRL10/RetNet/blob/main/char.ipynb

[Jamie-Stirling]

There's now a file example.py which outputs the number of parameters when hyperparameters are equal to those for the 1.3B model in the original paper.

[thegodone]

Great thanks a lot.

Could you also provide similar harry potter demo see my link https://github.com/DonRL10/RetNet/blob/main/char.ipynb ?