-
Notifications
You must be signed in to change notification settings - Fork 41
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #58 from projectatomic/i31
fix: separate models from recipes
- Loading branch information
Showing
9 changed files
with
229 additions
and
56 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| { | ||
| "recipes": [ | ||
| { | ||
| "id": "chatbot", | ||
| "description" : "Chat bot application", | ||
| "name" : "ChatBot", | ||
| "repository": "https://github.com/axel7083/locallm", | ||
| "icon": "natural-language-processing", | ||
| "categories": [ | ||
| "natural-language-processing" | ||
| ], | ||
| "config": "chatbot/ai-studio.yaml", | ||
| "readme": "# Locallm\n\nThis repo contains artifacts that can be used to build and run LLM (Large Language Model) services locally on your Mac using podman. These containerized LLM services can be used to help developers quickly prototype new LLM based applications, without the need for relying on any other externally hosted services. Since they are already containerized, it also helps developers move from their prototype to production quicker. \n\n## Current Locallm Services: \n\n* [Chatbot](#chatbot)\n* [Text Summarization](#text-summarization)\n* [Fine-tuning](#fine-tuning)\n\n### Chatbot\n\nA simple chatbot using the gradio UI. Learn how to build and run this model service here: [Chatbot](/chatbot/).\n\n### Text Summarization\n\nAn LLM app that can summarize arbitrarily long text inputs. Learn how to build and run this model service here: [Text Summarization](/summarizer/).\n\n### Fine Tuning \n\nThis application allows a user to select a model and a data set they'd like to fine-tune that model on. Once the application finishes, it outputs a new fine-tuned model for the user to apply to other LLM services. Learn how to build and run this model training job here: [Fine-tuning](/finetune/).\n\n## Architecture\n\n\nThe diagram above indicates the general architecture for each of the individual model services contained in this repo. The core code available here is the \"LLM Task Service\" and the \"API Server\", bundled together under `model_services`. With an appropriately chosen model downloaded onto your host,`model_services/builds` contains the Containerfiles required to build an ARM or an x86 (with CUDA) image depending on your need. These model services are intended to be light-weight and run with smaller hardware footprints (given the Locallm name), but they can be run on any hardware that supports containers and scaled up if needed.\n\nWe also provide demo \"AI Applications\" under `ai_applications` for each model service to provide an example of how a developers could interact with the model service for their own needs. ", | ||
| "models": [ | ||
| "llama-2-7b-chat.Q5_K_S", | ||
| "albedobase-xl-1.3", | ||
| "sdxl-turbo" | ||
| ] | ||
| } | ||
| ], | ||
| "models": [ | ||
| { | ||
| "id": "llama-2-7b-chat.Q5_K_S", | ||
| "name": "Llama-2-7B-Chat-GGUF", | ||
| "description": "Llama 2 is a family of state-of-the-art open-access large language models released by Meta today, and we’re excited to fully support the launch with comprehensive integration in Hugging Face. Llama 2 is being released with a very permissive community license and is available for commercial use. The code, pretrained models, and fine-tuned models are all being released today 🔥", | ||
| "hw": "CPU", | ||
| "registry": "Hugging Face", | ||
| "popularity": 3, | ||
| "license": "?", | ||
| "url": "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_S.gguf" | ||
| }, | ||
| { | ||
| "id": "albedobase-xl-1.3", | ||
| "name": "AlbedoBase XL 1.3", | ||
| "description": "Stable Diffusion XL has 6.6 billion parameters, which is about 6.6 times more than the SD v1.5 version. I believe that this is not just a number, but a number that can lead to a significant improvement in performance. It has been a while since we realized that the overall performance of SD v1.5 has improved beyond imagination thanks to the explosive contributions of our community. Therefore, I am working on completing this AlbedoBase XL model in order to optimally reproduce the performance improvement that occurred in v1.5 in this XL version as well. My goal is to directly test the performance of all Checkpoints and LoRAs that are publicly uploaded to Civitai, and merge only the resources that are judged to be optimal after passing through several filters. This will surpass the performance of image-generating AI of companies such as Midjourney. As of now, AlbedoBase XL v0.4 has merged exactly 55 selected checkpoints and 138 LoRAs.", | ||
| "hw": "CPU", | ||
| "registry": "Civital", | ||
| "popularity": 3, | ||
| "license": "openrail++", | ||
| "url": "" | ||
| }, | ||
| { | ||
| "id": "sdxl-turbo", | ||
| "name": "SDXL Turbo", | ||
| "description": "SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one.", | ||
| "hw": "CPU", | ||
| "registry": "Hugging Face", | ||
| "popularity": 3, | ||
| "license": "sai-c-community", | ||
| "url": "" | ||
| } | ||
| ], | ||
| "categories": [ | ||
| { | ||
| "id": "natural-language-processing", | ||
| "name": "Natural Language Processing", | ||
| "description" : "Models that work with text: classify, summarize, translate, or generate text." | ||
| }, | ||
| { | ||
| "id": "computer-vision", | ||
| "description" : "Process images, from classification to object detection and segmentation.", | ||
| "name" : "Computer Vision" | ||
| }, | ||
| { | ||
| "id": "audio", | ||
| "description" : "Recognize speech or classify audio with audio models.", | ||
| "name" : "Audio" | ||
| }, | ||
| { | ||
| "id": "multimodal", | ||
| "description" : "Stuff about multimodal models goes here omg yes amazing.", | ||
| "name" : "Multimodal" | ||
| } | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| /********************************************************************** | ||
| * Copyright (C) 2024 Red Hat, Inc. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| * | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| ***********************************************************************/ | ||
|
|
||
| /* eslint-disable @typescript-eslint/no-explicit-any */ | ||
|
|
||
| import { expect, test, vi } from 'vitest'; | ||
| import content from './ai-test.json'; | ||
| import { ApplicationManager } from './managers/applicationManager'; | ||
| import { RecipeStatusRegistry } from './registries/RecipeStatusRegistry'; | ||
| import { StudioApiImpl } from './studio-api-impl'; | ||
| import { PlayGroundManager } from './playground'; | ||
| import { TaskRegistry } from './registries/TaskRegistry'; | ||
|
|
||
| vi.mock('./ai.json', () => { | ||
| return { | ||
| default: content | ||
| }; | ||
| }); | ||
|
|
||
| const studioApiImpl = new StudioApiImpl( | ||
| {} as unknown as ApplicationManager, | ||
| {} as unknown as RecipeStatusRegistry, | ||
| {} as unknown as TaskRegistry, | ||
| {} as unknown as PlayGroundManager, | ||
| ) | ||
|
|
||
| test('expect correct model is returned with valid id', async () => { | ||
| const model = await studioApiImpl.getModelById('llama-2-7b-chat.Q5_K_S'); | ||
| expect(model).toBeDefined(); | ||
| expect(model.name).toEqual('Llama-2-7B-Chat-GGUF'); | ||
| expect(model.registry).toEqual('Hugging Face'); | ||
| expect(model.url).toEqual('https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_S.gguf') | ||
| }); | ||
|
|
||
| test('expect error if id does not correspond to any model', async () => { | ||
| await expect(() => studioApiImpl.getModelById('unknown')).rejects.toThrowError( | ||
| 'No model found having id unknown', | ||
| ); | ||
| }) | ||
|
|
||
| test('expect array of models based on list of ids', async () => { | ||
| const models = await studioApiImpl.getModelsByIds(['llama-2-7b-chat.Q5_K_S', 'albedobase-xl-1.3']); | ||
| expect(models).toBeDefined(); | ||
| expect(models.length).toBe(2); | ||
| expect(models[0].name).toEqual('Llama-2-7B-Chat-GGUF'); | ||
| expect(models[0].registry).toEqual('Hugging Face'); | ||
| expect(models[0].url).toEqual('https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_S.gguf'); | ||
| expect(models[1].name).toEqual('AlbedoBase XL 1.3'); | ||
| expect(models[1].registry).toEqual('Civital'); | ||
| expect(models[1].url).toEqual(''); | ||
| }); | ||
|
|
||
| test('expect empty array if input list is empty', async () => { | ||
| const models = await studioApiImpl.getModelsByIds([]); | ||
| expect(models).toBeDefined(); | ||
| expect(models.length).toBe(0); | ||
| }); | ||
|
|
||
| test('expect empty array if input list has ids that are not in the catalog', async () => { | ||
| const models = await studioApiImpl.getModelsByIds(['1', '2']); | ||
| expect(models).toBeDefined(); | ||
| expect(models.length).toBe(0); | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.