-
Notifications
You must be signed in to change notification settings - Fork 225
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' of https://github.com/Agenta-AI/agenta into featu…
…re/cloud_188-handle-large-datasets-for-open-api
- Loading branch information
Showing
36 changed files
with
507 additions
and
312 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| --- | ||
| title: "agenta init" | ||
| description: "Initialize a new project" | ||
| --- | ||
|
|
||
|
|
||
| ```bash | ||
| agenta init | ||
| ``` | ||
|
|
||
| # Description | ||
|
|
||
|
|
||
| The init command initializes a new agenta project. Users can define the application's name and select the agenta platform host (either local or cloud). | ||
|
|
||
| This command creates a blank project in the backend and generates a config.toml file for configuration settings. | ||
|
|
||
| <Info>This command creates a `project.toml` file containing all the information about your project</Info> | ||
|
|
||
| # Usage | ||
|
|
||
| When running `agenta init` you will prompted the following questions: | ||
|
|
||
| ### Project name | ||
| ```bash | ||
| ? Please enter the app name | ||
| ``` | ||
|
|
||
| Enter the app/project name | ||
|
|
||
| ### Agenta host | ||
| ```bash | ||
| ? Where are you running agenta? (Use arrow keys) | ||
| » On agenta cloud | ||
| On my local machine | ||
| On a remote machine | ||
| ``` | ||
|
|
||
| Here you can select where the agenta platform is hosted. | ||
| - Select cloud, in case you are using agenta cloud ([https://cloud.agenta.ai](https://cloud.agenta.ai)). In that case you will prompted to enter your API key. You can get these from the [agenta cloud configuration](https://cloud.agenta.ai/settings?tab=apiKeys). | ||
| - Select local machine if you are running the OSS version of agenta on your local machine ([http://localhost](http://localhost)) | ||
| - Select remote machine, in case you are running the OSS version of agenta on a remote machine. In that case you will prompted to enter the remote machine URL. | ||
|
|
||
| <Info> In case you are running agenta enterprise, please refer to the enterprise documentation on how to set up your agenta project</Info> | ||
|
|
||
| ### Start from scratch or from a template | ||
| ```bash | ||
| ? How do you want to initialize your app? (Use arrow keys) | ||
| » Blank App | ||
| Start from template | ||
| ``` | ||
|
|
||
| Chose blank app to start from scratch (write your own code) or start from template to use one of the agenta templates. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| title: "Install" | ||
| description: "Agenta's CLI enables developers to easily create, evaluate, and deploy LLM apps from code. Install it here" | ||
| --- | ||
|
|
||
| # Installation | ||
|
|
||
| The agenta CLI can be easily installed through pip: | ||
|
|
||
| ```bash | ||
| pip install -U agenta | ||
| ``` | ||
|
|
||
| # Quick usage guide | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Quick Start" icon="circle-play" href="/cli/quick-usage"> | ||
| Get an overview of the main commands and capabilities of agenta CLI | ||
| </Card> | ||
| <Card title="Tutorial" icon="books" href="/quickstart/getting-started-code"> | ||
| Jump into a tutorial deploying an LLM app from code using agenta CLI | ||
|
|
||
| </Card> | ||
| </CardGroup> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| --- | ||
| title: "Quick usage" | ||
| description: "Deploy and test LLM apps using agenta CLI" | ||
| --- | ||
|
|
||
|
|
||
| The Agenta CLI is a tool used to manage LLM app variants used in Agenta. It allows you to create new apps and deploy variants to the Agenta platform. | ||
|
|
||
| ## Create an application / project | ||
|
|
||
| To use the agenta CLI, first create a new folder for each project / application. | ||
|
|
||
| ```bash | ||
| mkdir my_app | ||
| ``` | ||
|
|
||
| Next, initialize the project in agenta by running | ||
|
|
||
| ```bash | ||
| agenta init | ||
| ``` | ||
|
|
||
| agenta init creates an empty project in the agenta platform (or populates it on one of the templates). | ||
|
|
||
| ## Write the code [optional] | ||
| Depending whether you initialized the project with a template or not, you may need to write the code for your variant. | ||
| The code for your new variant in a .py file. The file should contain a function marked with the `@ag.entrypoint` decorator. | ||
|
|
||
| Here is an example | ||
| ```python | ||
|
|
||
| import agenta as ag | ||
|
|
||
| ag.config.register_default(prompt="Translate {sentence} to {language}) | ||
|
|
||
| @ag.entrypoint | ||
| def translate(sentence:str, language:str): | ||
| ### add here openai call logic | ||
| ``` | ||
| ## Serve the application | ||
|
|
||
| ```bash | ||
| agenta variant serve myapp.py | ||
| ``` | ||
|
|
||
| This command deploys a new variant to the Agenta platform. It processes the code in the specified folder, with `myapp.py` as the entrypoint. This command builds a Docker image and deploys a container based on it. As a result, the variant becomes accessible in the web UI, allowing for prediction generation and API calls. The variant is named as `myapp.default` in the UI. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| title: "agenta variant list" | ||
| description: "List the variants for an application" | ||
| --- | ||
|
|
||
| ```bash | ||
| agenta variant list | ||
| ``` | ||
|
|
||
| # Description | ||
| The `list` command displays all the variants of your app that are currently available in the backend. | ||
|
|
||
| <Warning> The `variant list` command can only be run in a directory where the `config.toml` generated by `agenta init` is present. </Warning> | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| --- | ||
| title: "agenta variant remove" | ||
| description: "Removes a variant from an app/project" | ||
| --- | ||
|
|
||
| ```bash | ||
| agenta variant remove | ||
| ``` | ||
|
|
||
| # Description | ||
|
|
||
| `variant remove` removes a variant from an app/project. It is called without any argument. The list of variants in the app/project is displayed and the user is prompted to select one of them to remove. | ||
|
|
||
|
|
||
| <Warning> The `variant remove` command can only be run in a directory where the `config.toml` generated by `agenta init` is present. </Warning> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| title: "agenta variant serve" | ||
| description: "Serve an application to the agenta platform" | ||
| --- | ||
|
|
||
| ```bash | ||
| agenta variant serve app_name.py | ||
| ``` | ||
|
|
||
| # Description | ||
| The `serve` command deploys the code of an app to the agenta platform. The command packages the code in the .py file along with any additional files in the same directory and deploy them on the agenta platform. Once deployed, the variant becomes accessible the web UI. It can ehtn be further deployed to a staging or production environment as an API endpoint. | ||
|
|
||
| In the agenta UI, the deployed variant is initially named `app_name.default` where `app_name` is the name of the python file deployed and `default` the default configuration. Creating a new configuration `someconfig` (either from the UI or from CLI), will result in the creation of corresponding new variant, named `app_name.someconfig`. | ||
|
|
||
|
|
||
| <Warning> The code in `app_name.py` needs to include an entrypoint function. This function should be marked with the `@agenta.entrypoint` decorator. </Warning> | ||
|
|
||
| Below is a brief example of a valid `app_name.py` file: | ||
|
|
||
| ```python | ||
|
|
||
| import agenta as ag | ||
|
|
||
| ag.config.register_default(prompt="Translate {sentence} to {language}) | ||
|
|
||
| @ag.entrypoint | ||
| def translate(sentence:str, language:str): | ||
| ### add here openai call logic | ||
| ``` | ||
|
|
||
| <Warning> The `variant serve` command can only be run in a directory where the `config.toml` generated by `agenta init` is present.</Warning> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| --- | ||
| title: How to use a Custom LLM in agenta | ||
| description: 'Learn how to write an LLM application that uses a custom LLM' | ||
| --- | ||
|
|
||
| Using a custom LLM in Agenta is straightforward. The process involves writing the code for a custom application using the SDK, which then calls the LLM. | ||
|
|
||
| Below is the structure of a custom application that calls a [vllm hosted model on an API server](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#api-server): | ||
|
|
||
| ```python | ||
|
|
||
| import agenta as ag | ||
| import requests | ||
|
|
||
| default_prompt = "Please write a joke about {subject}" | ||
|
|
||
| url = "https://<api-server-url>/generate" | ||
| ag.config.default(prompt=default_prompt, | ||
| temperature=0.8) | ||
|
|
||
| @ag.entrypoint | ||
| def generate(subject:str)->str: | ||
| prompt = config.prompt.format(subject=subject) | ||
| data = { | ||
| "prompt": prompt, | ||
| "temperature": config.temperature | ||
| } | ||
| response = requests.post(url, data=json.dumps(data)) | ||
| return response.json() | ||
| ``` | ||
|
|
||
| The above code is a simple LLM app that generates jokes about a given subject, using a vLLM hosted model. It is structured as follows: | ||
|
|
||
| `ag.config.default` sets the default values for the configuration of the LLM application. In this example, the default prompt is "Please write a joke about {subject}", and the temperature is set at 0.8. | ||
|
|
||
| The `@ag.entrypoint` decorator marks the function that will be called. The function `generate` accepts a subject as input and returns a joke as output. It calls the vLLM hosted model using the requests library. | ||
|
|
||
| To call any other LLM, you need to set up the configuration for the LLM (prompt, temperature, etc.) and then call the LLM in the main function. | ||
|
|
||
| After writing the code, it can be deployed using the CLI, as described in the [command line reference](/cli/quick-usage). This can be done by running `agenta init` followed by `agenta variant serve app.py` in the code folder. | ||
|
|
||
| <Warning> Note that if the LLM is hosted on your local machine and not accessible from outside, you will need to [self-host agenta locally](self-host/host-locally) to be able to call the LLM from the LLm app. </Warning> |
Oops, something went wrong.