ml-toolkit

This package is for me to have a central and standardized library for designing experiments and collecting data.

Installation

create a virtual environment using python 3.11
run pip install -e . in the project root directory
- this allows you to execute the code by just calling mltoolkit when your virtual environment is enabled
go through the setup process for the HuggingFace Accelerate Library by running accelerate config
to train a model, use the command accelerate launch --no_python mltk -c <<config_path>>
- NOTE: you can just call mltk -c <<config_path>> but the program will not be using any GPUs so this would be used moreso for testing

Browse the configuration files directory to get an overview of the available models and how to train them