From 8f3fc29dd4df0084c7d4e7049051b5913217be6b Mon Sep 17 00:00:00 2001 From: Dividor Date: Wed, 10 Jul 2024 19:57:05 -0400 Subject: [PATCH 1/2] Documentation updates --- CONTRIBUTION.md | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/CONTRIBUTION.md b/CONTRIBUTION.md index 9b1b7347..29aa7acb 100644 --- a/CONTRIBUTION.md +++ b/CONTRIBUTION.md @@ -24,6 +24,10 @@ in the paragraphs below. The easiest way to develop is to run in the Docker environment, see [README](./README.md) for more details. +### Resetting your environment + +If running locally, you can reset your environment - removing any data for your databases, which means re-registration - by running `./cleanup.sh`. + ## Code quality tests The repo has been set up with black and flake8 pre-commit hooks. These are configured in the ``.pre-commit-config.yaml` file and initialized with `pre-commit autoupdate`. @@ -32,6 +36,8 @@ On a new repo, you must run `pre-commit install` to add pre-commit hooks. To run code quality tests, you can run `pre-commit run --all-files` +GitHub has an action to run the pre-commit tests to ensure code adheres to standards. See folder `'github/workflows` for more details. + ## Tests You should write tests for every feature you add or bug you solve in the code. @@ -47,10 +53,6 @@ the desired feature. You can use `pytest` to run your tests, no matter which type of test it is. -### Code quality tests - -GitHub has an action to run the pre-commit tests to ensure code adheres to standards. See folder `'github/workflows` for more details. - ### End-to-end tests End-to-end tests have been configured in GitHub actions which use promptflow to call a wrapper around the chainlit UI, or order to test when memories/recipes are used as well as when the assistant does some on-the-fly analysis. To do this, the chainlit class is patched heavily, and there are limitations in how @@ -147,15 +149,6 @@ To download demo data ... 2. `cd data && python3 download_demo_data.py && cd ..` 3. `docker compose start datadb` -## Misc. - -### Testing connection to actions server - -1. `docker exec -it haa-libre-chat /bin/sh` -2. To test the SQL query action, run `curl -X POST -H "Content-Type: application/json" -d '{"query": "select 1"}' "http://actions:8080/api/actions/postgresql-universal-actions/execute-query/run"` -3. To get get-memory action, run ... `curl -X POST -H "Content-Type: application/json" -d '{"chat_history": "[]", "user_input":"population of Mali", "generate_intent":"true"}' "http://actions:8080/api/actions/get-data-recipe-memory/get-memory-recipe/run"` - - # Evaluation with Prompt Flow First, you will need to build the environment to include Prompt Flow ... @@ -200,4 +193,4 @@ Note: :warning: *This is very much a work in progress, deployment will be automated with fewer compose files soon* -You will need to set key environment variables, see your local `.env` for examples. \ No newline at end of file +You will need to set key environment variables, see your local `.env` for examples. From b88ff1f23f1ba7f2821436cc92ee00d7687cd814 Mon Sep 17 00:00:00 2001 From: Dividor Date: Wed, 10 Jul 2024 20:01:04 -0400 Subject: [PATCH 2/2] Updated CLI README to use docker exec --- README.md | 72 ++++++++++++++++++------------------------------------- 1 file changed, 23 insertions(+), 49 deletions(-) diff --git a/README.md b/README.md index 7176c578..7a7961a2 100644 --- a/README.md +++ b/README.md @@ -136,7 +136,7 @@ We are in a phase of research to identify and improve recipes, but for now the s ### Adding your own files for the assistant to analyze -The assistant can be configured to analyze your own files, either in searching them or using them when analyzing data on-the-fly. To add your won files, place them in one of the following folders: +The assistant can be configured to analyze your own files, either in searching them or using them when analyzing data on-the-fly. To add your own files, place them in one of the following folders: `./assistants/chat_ui/files/file_search/custom` : The assistant will search these files `./assistants/chat_ui/files/code_interpreter/custom` : The assistant can use these files when generating and running code @@ -173,33 +173,15 @@ Then run ingestion in download only mode ... 5. `python ingest.py --skip_processing --skip_uploading` - -# To start the environment - -You can also access the recipe server monitoring endpoint: - -- Recipes server: [http://localhost:4001/](http://localhost:4001/) - -## Resetting your environment - -If running locally, you can reset your environment - removing any data for your databases, which means re-registration - by running `./cleanup.sh`. - -# Development - -## Managing recipes +# Managing recipes The management of recipes is part of the human in the loop approach of this repo. New recipes are created in status pending and only get marked as approved, once they have been verified by a recipe manager. Recipe managers can 'check out' recipes from the database into their local development environment such as VS Code to run, debug, and edit the recipes, before checking them back in. To make this process platform independent, recipes are checked out into a docker container, which can be used as the runtime environment to run the recipes via VSCode. -Recipes are managed using the recipes Command Line Interface (CLI), which allows you to check out recipes, run and refine, the commit them back to the recipes database for use in data recipes AI. - -To run the cli, you will need to install some packages ... - -`pip3 install typer` +Recipes are managed using the recipes Command Line Interface (CLI), which allows you to check out recipes, run and refine with LLM assistance, then commit them back to the recipes database for use in data recipes AI. -Once this is done, and you have your docker environment running as described above, you start the recipes CLI with ... +To run the CLI, you will need to start the docker environment as described in the 'Quick Start', then -`cd management` -`python cli.py` +`docker compose exec -it manager python cli.py` When you first log in, you will be asked for your name. This is used when checking in recipes. Once in, you will be presented with a menu like this ... @@ -208,16 +190,23 @@ When you first log in, you will be asked for your name. This is used when checki Welcome to the recipes management CLI, matt! Here are the commands you can run: - - 'checkout': Check out recipes for you to work on - 'list': List all recipes that are checked out - 'run': Run a recipe, you will be prompted to choose which one - 'add': Add a new recipe - 'delete': Delete a recipe, you will be prompted to choose which one - 'checkin': Check in recipes you have completed - 'makemem': Create a memory using recipe sample output - 'help': Show a list of commands - 'quit': Exit this recipes CLI + + 'checkout': Check out recipes for you to work on + 'list': List all recipes that are checked out + 'run': Run a recipe, you'll be prompted, or use 'run 5' to run 5. + 'add': Add a new recipe (using LLM) + 'edit': Edit a recipe (using LLM). You'll be prompted, or use 'edit 5' to edit 5. + 'delete': Delete a recipe, you will be prompted to choose which one + 'checkin': Check in recipes you have completed + 'makemem': Create a memory using recipe sample output + 'rebuild': Removes database data, runs all local recipes and checks them in + 'dumpdb': Dump embedding, recipe and memory tables to DB upgrade files so included in build + 'help': Show a list of commands + 'quit': Exit this recipes CLI + + Chat with Data Mode: + + 'chat': Enter data chat mode to ask questions about the data Type one of the commands above to do some stuff. @@ -225,7 +214,7 @@ Welcome to the recipes management CLI, matt! >> ``` -The first thing you will want to do is run 'checkout' to get all the recipe code from the database onto your computer so you can run them. Once you have them locally, you can edit them in tools like Visual Studio code. +The first thing you will want to do is run 'checkout' to get all the recipe code from the database onto your computer so you can run them. Once you have them locally, you can edit them in tools like Visual Studio code. They will appear in folder './management/work'. To run recipes locally you can use the CLI 'run' command. This will run the recipe in the same environment, and will save the results like sample outputs, for you so they can be published back to the database. @@ -233,21 +222,6 @@ You can create new recipes by entering 'add', where you'll be prompted for an in When ready, you can check in your new and edited recipes with 'checkin'. -### Other approaches - -You can also configure VS Code to connect to the recipe-manage container for running recipes ... - -1. Install the DevContainers VSCode extension -2. Build data recipes using the `docker compose` command mentioned above -3. Open the command palette in VSCode (CMD + Shift + P on Mac; CTRL + Shift + P on Windows) and select - - `Dev Containers: Attach to remote container`. - - Select the recipe-manager container. This opens a new VSCode window - use it for the next steps. -4. Open folder `/app` -5. Navigate to your recipe in sub-folder `checked_out` -6. Run the `recipe.py` in a terminal or set up the docker interpretor - # Autogen Studio and autogen agent teams for creating data recipes ![alt text](./assets/autogen-studio-recipes.png)