Skip to content

Commit

Permalink
restructure
Browse files Browse the repository at this point in the history
  • Loading branch information
mmabrouk committed May 10, 2024
1 parent b4b3bfb commit 852d369
Show file tree
Hide file tree
Showing 22 changed files with 156 additions and 60 deletions.
5 changes: 0 additions & 5 deletions docs/advanced_guides/using_custom_evaluators.mdx

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
Empty file added docs/evaluation/overview.mdx
Empty file.
File renamed without changes.
14 changes: 7 additions & 7 deletions docs/getting_started/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ Agenta is an open-source platform that helps **developers** and **product teams*

### With Agenta, you can:

1. Rapidly [**experiment** and **compare** prompts](/basic_guides/prompt_engineering) on [any LLM workflow](/advanced_guides/custom_applications) (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
2. Rapidly [**create test sets**](/basic_guides/test_sets) and **golden datasets** for evaluation
1. Rapidly [**experiment** and **compare** prompts](/prompt_management/prompt_engineering) on [any LLM workflow](/prompt_management/custom_applications) (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
2. Rapidly [**create test sets**](/evaluation/test_sets) and **golden datasets** for evaluation
3. **Evaluate** your application with pre-existing or **custom evaluators**
4. **Annotate** and **A/B test** your applications with **human feedback**
5. [**Collaborate with product teams**](/basic_guides/team_management) for prompt engineering and evaluation
6. [**Deploy your application**](/basic_guides/deployment) in one-click in the UI, through CLI, or through github workflows.
5. [**Collaborate with product teams**](/misc/team_management) for prompt engineering and evaluation
6. [**Deploy your application**](/prompt_management/deployment) in one-click in the UI, through CLI, or through github workflows.

Agenta focuses on increasing the speed of the development cycle of LLM applications by increasing the speed of experimentation.

Expand All @@ -23,7 +23,7 @@ Agenta focuses on increasing the speed of the development cycle of LLM applicati
### Works with any LLM app workflow
Agenta enables prompt engineering and evaluation on any LLM app architecture, such as **Chain of Prompts**, **RAG**, or **LLM agents**. It is compatible with any framework like **Langchain** or **LlamaIndex**, and works with any model provider, such as **OpenAI**, **Cohere**, or **local models**.

[Jump here](/advanced_guides/custom_applications) to see how to use your own custom application with Agenta and [here](/developer_guides/how_does_agenta_work) to understand more how Agenta works.
[Jump here](/prompt_management/custom_applications) to see how to use your own custom application with Agenta and [here](/guides/how_does_agenta_work) to understand more how Agenta works.

### Enable collaboration between developers and product teams

Expand All @@ -39,7 +39,7 @@ By **adding a few lines to your application code**, you can create a prompt play
<Card
title="Get Started"
icon="stars"
href="/getting_started/getting-started-ui"
href="/getting_started/quick-start"
color="#33FF57">
Create and deploy your first app from the UI in under 2 minutes.
</Card>
Expand All @@ -48,7 +48,7 @@ By **adding a few lines to your application code**, you can create a prompt play
title="Create a Custom App"
icon="code"
color="#337BFF"
href="/advanced_guides/custom_applications">
href="/prompt_management/custom_applications">
Write a custom LLM app and evaluate it in 10 minutes.
</Card>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: 'Quick Start'
description: 'Create and deploy your first LLM app in one minute'
---

<Note>This tutorial helps users create LLM apps using templates within the UI. For more complex applications involving code in Agenta, please refer to Using code in Agenta [Using code in agenta](/advanced_guides/custom_applications) </Note>
<Note>This tutorial helps users create LLM apps using templates within the UI. For more complex applications involving code in Agenta, please refer to Using code in Agenta [Using code in agenta](/prompt_management/custom_applications) </Note>

Want a video tutorial instead? We have a 4-minute video for you. [Watch it here](https://youtu.be/plPVrHXQ-DU).

Expand Down Expand Up @@ -57,4 +57,4 @@ You can now find the API endpoint in the "Endpoints" menu. Copy and paste the co
<img height="600" src="/images/getting-started-ui-screenshots/06_deployment.png" />


<Check> Congratulations! You've created your first LLM application. Feel free to modify it, explore its parameters, and discover Agenta's features. Your next steps could include [building an application using your own code](/advanced_guides/custom_applications), or following one of our UI-based tutorials.</Check>
<Check> Congratulations! You've created your first LLM application. Feel free to modify it, explore its parameters, and discover Agenta's features. Your next steps could include [building an application using your own code](/prompt_management/custom_applications), or following one of our UI-based tutorials.</Check>
8 changes: 4 additions & 4 deletions docs/guides/list_templates.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,17 @@ description: "A collection of templates and tutorials indexed by architecture."

# Tutorials
## 📝 Text Generation
### [Single Prompt Application using OpenAI and Langchain](/developer_guides/tutorials/first-app-with-langchain)
### [Single Prompt Application using OpenAI and Langchain](/guides/tutorials/first-app-with-langchain)
<span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Text Generation </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> OpenAI </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Langchain </span><span>&nbsp;</span>

Learn how to use our SDK to deploy an application with agenta. The application we will create uses OpenAI and Langchain. The application generates outreach messages in Linkedin to investors based on a startup name and idea.
### [Use Mistral from Huggingface for a Summarization Task](/developer_guides/tutorials/deploy-mistral-model)
### [Use Mistral from Huggingface for a Summarization Task](/guides/tutorials/deploy-mistral-model)
<span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Text Generation </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Mistral </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Hugging Face </span><span>&nbsp;</span>

Learn how to use a custom model with agenta.

## Retrieval Augmented Generation (RAG)
### [RAG Application with LlamaIndex](/developer_guides/tutorials/build-rag-application)
### [RAG Application with LlamaIndex](/guides/tutorials/build-rag-application)
<span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Sales </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> OpenAI </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> RAG </span><span>&nbsp;</span><span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> LlamaIndex </span>

Learn how to create a RAG application with LlamaIndex and use it in agenta. You will create a playground in agenta where you can experiment with the parameters of the RAG application, test it and compare different versions.
Expand All @@ -24,7 +24,7 @@ Learn how to create a RAG application with LlamaIndex and use it in agenta. You
## ⛏️ Extraction

These templates extract data in a structured format from an unstructured source.
### [Extraction using OpenAI Functions and Langchain](/cookbook/extract_job_information)
### [Extraction using OpenAI Functions and Langchain](/guides/extract_job_information)

<span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> HR </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> OpenAI </span> <span>&nbsp;</span> <span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Functions </span><span>&nbsp;</span><span style={{ backgroundColor: '#FEF0F0', color: '#DC2626'}}> Langchain </span>

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/prompt_management/asaproxy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
44 changes: 15 additions & 29 deletions docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
"group": "Getting Started",
"pages": [
"getting_started/introduction",
"getting_started/getting-started-ui"
"getting_started/quick-start"
]
},
{
Expand All @@ -75,40 +75,29 @@
{
"group": "Setting up",
"pages": [
"basic_guides/creating_an_app",
"advanced_guides/custom_applications",
"advanced_guides/using_agenta_from_cli"
"prompt_management/creating_an_app",
"prompt_management/custom_applications",
"prompt_management/using_agenta_from_cli"
]
},
"basic_guides/prompt_engineering",
"basic_guides/deployment",
"basic_guides/integrating"
"prompt_management/prompt_engineering",
"prompt_management/deployment",
"prompt_management/integrating"
]
},
{
"group": "Evaluation",
"pages": [
"basic_guides/overview",
"basic_guides/test_sets",
"basic_guides/automatic_evaluation",
"basic_guides/custom_evaluator",
"basic_guides/human_evaluation"
"evaluation/test_sets",
"evaluation/automatic_evaluation",
"evaluation/custom_evaluator",
"evaluation/human_evaluation"
]
},
{
"group": "Monitoring",
"group": "Tracing and Monitoring",
"pages": [
{
"group": "Setting up",
"pages": [
"basic_guides/creating_an_app",
"advanced_guides/custom_applications",
"advanced_guides/using_agenta_from_cli"
]
},
"basic_guides/prompt_engineering",
"basic_guides/deployment",
"basic_guides/integrating"
"monitoring/overview"
]
},
{
Expand All @@ -130,11 +119,8 @@
{
"group": "Misc",
"pages": [
"basic_guides/team_management",
"basic_guides/getting_support",
"basic_guides/faq",
"basic_guides/privacy_policy",
"basic_guides/Open-source",
"misc/team_management",
"misc/getting_support",
{
"group": "Contributing",
"pages": [
Expand Down
2 changes: 1 addition & 1 deletion docs/misc/contributing/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ To maintain code quality, we adhere to certain formatting and linting rules:

## Contribution Steps

1. **Pick an Issue:** Start by selecting an issue from our issue tracker. Choose one that matches your skill set and begin coding. For more on this, read our [Creating an Issue Guide](/developer_guides/contributing/file-issue).
1. **Pick an Issue:** Start by selecting an issue from our issue tracker. Choose one that matches your skill set and begin coding. For more on this, read our [Creating an Issue Guide](/misc/contributing/file-issue).

2. **Fork & Pull Request:** Fork our repository, create a new branch, add your changes, and submit a pull request. Ensure your code aligns with our standards and includes appropriate unit tests.

Expand Down
31 changes: 31 additions & 0 deletions docs/misc/getting_support.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
title: Getting Support
---

We offer multiple channels to get support for Agenta.

<CardGroup cols={2}>
<Card
title="Join our Slack"
icon="slack"
href="https://join.slack.com/t/agenta-hq/shared_invite/zt-1zsafop5i-Y7~ZySbhRZvKVPV5DO_7IA">
Use the #support channel on Slack for any inquiries or assistance with Agenta. Paying customers have a private Slack channel for support.
</Card>
<Card
title="Schedule an onboarding call"
icon="phone"
href="https://cal.com/mahmoud-mabrouk-ogzgey/demo">
Book a call with a founder for one-on-one guidance on using Agenta.
</Card>
<Card
title="Create an issue in Github"
icon="github"
href="https://github.com/agenta-ai/agenta">
File a bug report or start a discussion in our Github repository.
</Card>
<Card
title="Chat Widget"
icon="intercom">
Use the chat widget on the bottom right of the screen in the documentation or in the cloud version to ask questions or report issues.
</Card>
</CardGroup>
File renamed without changes.
75 changes: 75 additions & 0 deletions docs/monitoring/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: Overview
---

What isn't measured cannot be improved. If you want to enhance your application's quality, cost, and performance, you first need to monitor it. Agenta offers a comprehensive monitoring system that lets you track application performance metrics over time.

For every request made through your LLM application, Agenta captures the input variables, configuration (like prompt templates), outputs, and crucial metadata such as cost, latency, and model name.

Agenta also provides tools to visualize and analyze this data on a dashboard. You can view request count, average latency, operational costs, and more. Additionally, you can filter requests to identify problematic ones and add them to your test sets.

## Setting up tracing

### Applications created from the UI
When creating an application from the UI, tracing is enabled by default. There's no need for setup; simply go to the observability view to see all the requests in the dashboard and in the traces view.

### Application created from code
If you're creating your own template from code or hosting your own code base and want to use Agenta for prompt management and tracing, you'll need to set up tracing yourself.
<Steps>
<Step title="First Step">
These are instructions or content that only pertain to the first step.
</Step>
<Step title="Second Step">
These are instructions or content that only pertain to the second step.
</Step>
<Step title="Third Step">
These are instructions or content that only pertain to the third step.
</Step>
</Steps>

Here's a quick guide on how to do it. You can find more details in the rest of the documentation.

<Steps>
<Step title="Install the Agenta Python SDK in your project">
Install our Python SDK using pip:
```bash
pip install agenta
```
or add it to your `requirements.txt`/`pyproject.toml`.. file.
</Step>
<Step title="Set the agenta environment variables">
```python
import os
os.environment["AGENTA_API_KEY"] = "your_api_key"
os.envivonment["AGENTA_APP_ID"] = "your_app_id"
os.environment["AGENTA_HOST"] = "https://cloud.agenta.ai"
```
You can find your API key in the configuration menu. As for the app ID, you can find it in the deployment menu.
</Step>
<Step title="Add the tracing decorators">
Decorate your functions with the `@ag.span` decorator to trace the function and the `@ag.trace` decorator to trace the entire application. Note here that we provide integrations for many client such as OpenAI. Here's an example:
```python
import agenta as ag
import openai

client = openai.Client()
ag.instrument_openai(client)

@ag.span
def myllmcall(country:str):
prompt = f"What is the capital of {country}"
response = client.chat.completions.create(
model='gpt-4',
messages=[
{'role': 'user', 'content': prompt},
],
)
return response.choices[0].text

#@ag.entrypoint # in case you are hosting the app in agenta
@ag.trace
def generate(country:str):
return myllmcall(country)
```
</Step>
</Steps>
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: 'Creating an LLM App'
---

<Note>You can create applications in Agenta either from the web interface or from code. This guide will focus on creating an application using a template from the UI. You can read more about creating a custom application using code [here](/advanced_guides/custom_applications)</Note>
<Note>You can create applications in Agenta either from the web interface or from code. This guide will focus on creating an application using a template from the UI. You can read more about creating a custom application using code [here](/prompt_management/custom_applications)</Note>

## Step-by-step Guide

Expand Down Expand Up @@ -39,4 +39,4 @@ Like the single prompt application, the chat application is based on the OpenAI

## Next steps

Now that you've created an application, you can learn how to do [prompt engineering in the playground](/basic_guides/prompt_engineering).
Now that you've created an application, you can learn how to do [prompt engineering in the playground](/prompt_management/prompt_engineering).
Loading

0 comments on commit 852d369

Please sign in to comment.