Model Merging UI

[hipsterusername]

Summary

Model merging (and related functions) allow users to manipulate the model weights of various models without directly training the model. This functionality is supported in Invoke-training with scripts, but does not have direct Interface elements to interact with these capabilities.

Supported capabilities:

• Base Model Merging: Selecting two models to merge the weights. This includes merging mechanisms like TIES/DARE (task models)
• LoRA + Base Model Merging: Selecting N LoRAs to merge with a Base Model
• Model Extraction: This selects a trained model, as well as a base model, and extracts the training as a LoRA.

Related scripts can be found in src/invoke_training/model_merge/scripts.

Users should have a new interface in addition to datasets/training that allows the user to leverage the above. The interface from the training tabs can be used as a base for UI layout/reference.

There are a lot of UX improvement opportunities generally across the training application (better template management, file path usage, etc.), which could be incorporated.