Skip to content

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.

License

Notifications You must be signed in to change notification settings

Agenta-AI/agenta

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Shows the logo of agenta

Documentation | Website | Slack

The Open source LLMOps Platform

Prompt playground, prompt management, evaluation, and observability

MIT license. Doc PRs welcome Contributors Last Commit Commits per month PyPI - Downloads


Try Agenta Live Demo






Documentation β€’ Changelog β€’ Website β€’ Agenta Cloud


What is Agenta?

Agenta is a platform for building production-grade LLM applications. It helps engineering and product teams create reliable LLM apps faster.

Agenta provides end-to-end tools for the entire LLMOps workflow: building (LLM playground, evaluation), deploying (prompt and configuration management), and monitoring (LLM observability and tracing).

Features

  • Prompt Playground: Experiment, iterate on prompts, and compare outputs from over 50 LLM models side by side (docs)
  • Custom Workflows: Build a playground for any custom LLM workflow, such as RAG or agents. Enable all the team to easily iterate on its parameters and evaluate it from the web UI.
  • LLM evaluation: Run evaluation suite from the webUI using predefined evaluators like LLM-as-a-judge, RAG evaluators, or custom code evaluators. (docs)
  • Human evaluation: Collaborate with subject matter experts for human annotation evaluation, including A/B testing and annotating golden test sets.
  • Prompt Management: Version your prompts and manage them across different environments (docs, quick start)
  • LLM Tracing: Observe and debug your apps with integrations to most providers and frameworks (docs, quick start)
  • LLM Monitoring: Track cost and latency and compare different deployments.

Getting Started

Agenta Cloud:

The easiest way to get started is through Agenta Cloud. It is free to signup, and comes with a generous free-tier.

Get Started with Agenta Cloud

Self-host:

mkdir agenta && cd agenta
curl -L https://raw.githubusercontent.com/agenta-ai/agenta/main/docker-compose.gh.yml -o docker-compose.gh.yml
docker compose -f docker-compose.gh.yml up -d
  • Agenta uses port 80, to use a different port set the env var AGENTA_PORT. For instance export AGENTA_PORT=90
  • Add --pull always to the last command to upgrade your version of Agenta.

Disabling Anonymized Tracking

By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information. To disable anonymized telemetry, follow these steps:

  • For web: Set TELEMETRY_TRACKING_ENABLED to false in your agenta-web/.env file.
  • For CLI: Set telemetry_tracking_enabled to false in your ~/.agenta/config.toml file.

Contributing

We warmly welcome contributions to Agenta. Feel free to submit issues, fork the repository, and send pull requests.

We are usually hanging in our Slack. Feel free to join our Slack and ask us anything

Check out our Contributing Guide for more information.

Contributors ✨

All Contributors

Thanks goes to these wonderful people (emoji key):

Sameh Methnani
Sameh Methnani

πŸ’» πŸ“–
Suad Suljovic
Suad Suljovic

πŸ’» 🎨 πŸ§‘β€πŸ« πŸ‘€
burtenshaw
burtenshaw

πŸ’»
Abram
Abram

πŸ’» πŸ“–
Israel Abebe
Israel Abebe

πŸ› 🎨 πŸ’»
Master X
Master X

πŸ’»
corinthian
corinthian

πŸ’» 🎨
Pavle Janjusevic
Pavle Janjusevic

πŸš‡
Kaosi Ezealigo
Kaosi Ezealigo

πŸ› πŸ’»
Alberto Nunes
Alberto Nunes

πŸ›
Maaz Bin Khawar
Maaz Bin Khawar

πŸ’» πŸ‘€ πŸ§‘β€πŸ«
Nehemiah Onyekachukwu Emmanuel
Nehemiah Onyekachukwu Emmanuel

πŸ’» πŸ’‘ πŸ“–
Philip Okiokio
Philip Okiokio

πŸ“–
Abhinav Pandey
Abhinav Pandey

πŸ’»
Ramchandra Warang
Ramchandra Warang

πŸ’» πŸ›
Biswarghya Biswas
Biswarghya Biswas

πŸ’»
Uddeepta Raaj Kashyap
Uddeepta Raaj Kashyap

πŸ’»
Nayeem Abdullah
Nayeem Abdullah

πŸ’»
Kang Suhyun
Kang Suhyun

πŸ’»
Yoon
Yoon

πŸ’»
Kirthi Bagrecha Jain
Kirthi Bagrecha Jain

πŸ’»
Navdeep
Navdeep

πŸ’»
Rhythm Sharma
Rhythm Sharma

πŸ’»
Osinachi Chukwujama
Osinachi Chukwujama

πŸ’»
θŽ«ε°”η΄’
θŽ«ε°”η΄’

πŸ“–
Agunbiade Adedeji
Agunbiade Adedeji

πŸ’»
Emmanuel Oloyede
Emmanuel Oloyede

πŸ’» πŸ“–
Dhaneshwarguiyan
Dhaneshwarguiyan

πŸ’»
Priyanshu Prajapati
Priyanshu Prajapati

πŸ“–
Raviteja
Raviteja

πŸ’»
Arijit
Arijit

πŸ’»
Yachika9925
Yachika9925

πŸ“–
Aldrin
Aldrin

⚠️
seungduk.kim.2304
seungduk.kim.2304

πŸ’»
Andrei Dragomir
Andrei Dragomir

πŸ’»
diego
diego

πŸ’»
brockWith
brockWith

πŸ’»
Dennis Zelada
Dennis Zelada

πŸ’»
Romain Brucker
Romain Brucker

πŸ’»
Heon Heo
Heon Heo

πŸ’»
Drew Reisner
Drew Reisner

πŸ’»
Ikko Eltociear Ashimine
Ikko Eltociear Ashimine

πŸ“–
Vishal Vanpariya
Vishal Vanpariya

πŸ’»
Youcef Boumar
Youcef Boumar

πŸ“–
LucasTrg
LucasTrg

πŸ’» πŸ›
Ashraf Chowdury
Ashraf Chowdury

πŸ› πŸ’»
jp-agenta
jp-agenta

πŸ’» πŸ›
Mr Unhappy
Mr Unhappy

πŸ› πŸš‡
Moreno Bonaventura
Moreno Bonaventura

πŸ›

This project follows the all-contributors specification. Contributions of any kind are welcome!