Code

Main to evaluations in backend#1187

Code

Merged

aakrem merged 169 commits intoevaluations-in-backendAgenta-AI/agenta:evaluations-in-backendfrom main-to-evaluations-in-backendAgenta-AI/agenta:main-to-evaluations-in-backendCopy head branch name to clipboardJan 11, 2024

+8-5

Commits

Commits on Dec 10, 2023

• Feat - introduce func response type

  aybruhm

  committed
  8be6ab6

• Update - improve execute_function and entrypoint functions

  aybruhm

  committed
  e9356b3

Commits on Dec 14, 2023

• Merge branch 'main' into gh/sdk-output-format-change

  aybruhm

  committed
  fe7180b

• Update - modified execute_function to make use of time.perf_counter and also cleanup imports and entrypoint function

  aybruhm

  committed
  993a455

• 🎨 Format - ran black

  aybruhm

  committed
  ecad1c2

• Update - refactor types for func execute in sdk

  aybruhm

  committed
  c686dfe

• Update - modified execute_function return response

  aybruhm

  committed
  c9a0cbc

• Feat - created example app that uses the new sdk output format

  aybruhm

  committed
  33c43e6

Commits on Dec 18, 2023

• Minor refactor

  mmabrouk

  authored
  df2dd20

Commits on Dec 19, 2023

• Feat - created openai_cost helper function

  aybruhm

  committed
  60f483f

• Update - change cost type to float from str

  aybruhm

  committed
  99a95e9

• Feat - implemented calculate_token_usage helper function

  aybruhm

  committed
  2a1f381

• Update - change cost type to float from str

  aybruhm

  committed
  590ad2d

• Update - import calculate_token_usage helper function in sdk/__init__.py module

  aybruhm

  committed
  f22104b

• Update - added cost to example app

  aybruhm

  committed
  40734db

• 🎨 Format - ran black

  aybruhm

  committed
  0e32ea5

• 🎨 Format - ran black

  aybruhm

  committed
  72366c1

Commits on Dec 21, 2023

• add costs source

  aakrem

  authored
  6b10ce3

• Merge pull request #1025 from Agenta-AI/gh/sdk-output-format-change

  Show description for a0a2005
  aakrem

  authored
  a0a2005

Commits on Dec 22, 2023

• feat: add cost, latency and usage to playground

  bekossy

  committed
  b5d4cdb

Commits on Dec 26, 2023

• Refactor - update templates return response

  aybruhm

  committed
  84dbeaf

• create cli client backend from openapi.json using fern

  

  devgenix

  authored and

  aybruhm

  committed
  fb904ef

• rewrite cli to use new client

  

  devgenix

  authored and

  aybruhm

  committed
  4d9070b

• format black

  

  devgenix

  authored and

  aybruhm

  committed
  c752a00

• rename folder and move to api

  

  devgenix

  authored and

  aybruhm

  committed
  745f6d7

• rename client to api.py

  

  devgenix

  authored and

  aybruhm

  committed
  1133341

• create exceptions

  

  devgenix

  authored and

  aybruhm

  committed
  62aad1e

• update imports path

  

  devgenix

  authored and

  aybruhm

  committed
  e8d1782

• initialie client properly

  

  devgenix

  authored and

  aybruhm

  committed
  4594d38

• fix validate api key

  

  devgenix

  authored and

  aybruhm

  committed
  ff55bb9

• initialise client in helper & variant_commans.py

  

  devgenix

  authored and

  aybruhm

  committed
  59b61eb

• set initialize client with host & api_key

  

  devgenix

  authored and

  aybruhm

  committed
  655cae9

• set default for api_key

  

  devgenix

  authored and

  aybruhm

  committed
  5602cd9

• correct request payload and get data

  

  devgenix

  authored and

  aybruhm

  committed
  8c6d8d9

• handle client request successfully

  

  devgenix

  authored and

  aybruhm

  committed
  cbc1a58

• fix initialise client

  

  devgenix

  authored and

  aybruhm

  committed
  c08d6fc

• ran format

  

  devgenix

  authored and

  aybruhm

  committed
  40044d7

• refactor initialising client

  

  devgenix

  authored and

  aybruhm

  committed
  da73ae9

• format

  

  devgenix

  authored and

  aybruhm

  committed
  758e74e

• fix adding variant to server

  

  devgenix

  authored and

  aybruhm

  committed
  dbb1da6

• Define Router class to handle use of trailing and non-trailish / in endpoints

  

  devgenix

  authored and

  aybruhm

  committed
  5a879ac

• Refactor Routers to use custom APIRouter

  

  devgenix

  authored and

  aybruhm

  committed
  7f8d80b

• move api folder into client

  

  devgenix

  authored and

  aybruhm

  committed
  0aa47b1

• define operation_id name for use in openapi.json

  

  devgenix

  authored and

  aybruhm

  committed
  1ce20c5

• update client import path

  

  devgenix

  authored and

  aybruhm

  committed
  077f29a

• remove client wrapper

  

  devgenix

  authored and

  aybruhm

  committed
  f3bb7f7

• Create Readme

  

  devgenix

  authored and

  aybruhm

  committed
  fc220f8

• regenerate Backend

  

  devgenix

  authored and

  aybruhm

  committed
  21fbb56

• ran format black

  

  devgenix

  authored and

  aybruhm

  committed
  535c42a

• Add step to add Security to openapi.json

  

  devgenix

  authored and

  aybruhm

  committed
  2deb0be

• Regenerate backend with authentication

  

  devgenix

  authored and

  aybruhm

  committed
  5e97246

• update client function to the correct client function

  

  devgenix

  authored and

  aybruhm

  committed
  f44b82b

• format black

  

  devgenix

  authored and

  aybruhm

  committed
  2188467

• increase timeout for build image

  

  devgenix

  authored and

  aybruhm

  committed
  cd06d09

• Update Readme.md with image desc

  

  devgenix

  authored and

  aybruhm

  committed
  0b4bbbb

• fix client func name

  

  devgenix

  authored and

  aybruhm

  committed
  8e9586f

• fix indent

  

  devgenix

  authored and

  aybruhm

  committed
  4db9d2a

• update url for openapi and fix formatting

  

  devgenix

  authored and

  aybruhm

  committed
  5518640

• modify function name for clarity

  

  devgenix

  authored and

  aybruhm

  committed
  39c6235

• Update main.py

  

  devgenix

  authored and

  aybruhm

  committed
  bc498a0

• Update LICENSE

  

  mmabrouk

  authored and

  aybruhm

  committed
  ef716b9

• Update docker-compose command in host-locally.mdx

  

  mmabrouk

  authored and

  aybruhm

  committed
  ccf28e7

• Update architecture documentation

  

  mmabrouk

  authored and

  aybruhm

  committed
  cb8b2f5

• Add new images for apps and configurations

  

  mmabrouk

  authored and

  aybruhm

  committed
  4d0d78e

• Add documentation for LLM app concepts and

  Show description for 2493d60
  

  mmabrouk

  authored and

  aybruhm

  committed
  2493d60

• Update "Getting Started" section in

  Show description for 00ae190
  

  mmabrouk

  authored and

  aybruhm

  committed
  00ae190

• Update title in getting-started-ui.mdx file

  

  mmabrouk

  authored and

  aybruhm

  committed
  9333dea

• Add documentation on how Agenta works

  

  mmabrouk

  authored and

  aybruhm

  committed
  43f10e2

• Update navigation groups and pages in mint.json

  

  mmabrouk

  authored and

  aybruhm

  committed
  3cd49cd

• Remove "learn/concepts" page from mint.json

  

  mmabrouk

  authored and

  aybruhm

  committed
  121b981

• Update Agenta's framework documentation

  

  mmabrouk

  authored and

  aybruhm

  committed
  fc17091

• Update card titles and links in introduction.mdx

  

  mmabrouk

  authored and

  aybruhm

  committed
  1e45eff

• Add docstring for APIRouter

  

  devgenix

  authored and

  aybruhm

  committed
  59334b7

• correct cloud url

  

  devgenix

  authored and

  aybruhm

  committed
  fb07284

• Correct Readme.md

  

  devgenix

  authored and

  aybruhm

  committed
  5aecd11

• Fix image paths in architecture and

  Show description for 2c3f4e4
  

  mmabrouk

  authored and

  aybruhm

  committed
  2c3f4e4

• add security spec for openapi

  

  devgenix

  authored and

  aybruhm

  committed
  13cd5c3

• simplify steps to generate client code

  

  devgenix

  authored and

  aybruhm

  committed
  4a7f77f

• fix override variant

  

  devgenix

  authored and

  aybruhm

  committed
  44f0f35

• Regenerate Backend

  

  devgenix

  authored and

  aybruhm

  committed
  184bf33

• set default timeout to 600

  

  devgenix

  authored and

  aybruhm

  committed
  dc4eeab

• remove openapi schema from oss

  

  devgenix

  authored and

  aybruhm

  committed
  e3e8211

• format

  

  devgenix

  authored and

  aybruhm

  committed
  1821a12

• added batch execution logic in auto evaluations

  

  MohammedMaaz

  authored and

  aybruhm

  committed
  5450ee5

• b

  

  mmabrouk

  authored and

  aybruhm

  committed
  5c28893

• Update README.md

  

  mmabrouk

  authored and

  aybruhm

  committed
  c90cea0

• creates contextProvider

  

  bekossy

  authored and

  aybruhm

  committed
  35876ae

• fix: type error

  

  bekossy

  authored and

  aybruhm

  committed
  83cce46

• feat: sync resultList

  

  bekossy

  authored and

  aybruhm

  committed
  b2bbe2d

• feat: single prompt comparison mode

  

  bekossy

  authored and

  aybruhm

  committed
  1a3e929

• made ChatInput props immutable | fixed side by side playground for chat

  

  MohammedMaaz

  authored and

  aybruhm

  committed
  c2b3cd7

• made chatinputs prop immutable | fixed state syncing in side by side playground

  

  MohammedMaaz

  authored and

  aybruhm

  committed
  3309119

• feat: rerun single button in ai critics

  

  bekossy

  authored and

  aybruhm

  committed
  115376d

• Update pyproject.toml

  

  mmabrouk

  authored and

  aybruhm

  committed
  5b7fb3a

• Add changelog and update navigation in mint.json

  

  mmabrouk

  authored and

  aybruhm

  committed
  30499ee

• Update file paths in links

  

  mmabrouk

  authored and

  aybruhm

  committed
  a49e91d

• Feat - implemented BinaryParam SDK type and BoolMeta class

  aybruhm

  committed
  a9678b6

• Update - override BinaryParam subschema

  aybruhm

  committed
  fe84849

• Update - determine bool type for playground use

  aybruhm

  committed
  d6a8b82

• Update - include parameter switch ui for boolean param type

  aybruhm

  committed
  12f2550

• 🎨 Format - ran black and prettier --write .

  aybruhm

  committed
  14d9672

• Cleanup - remove BinaryParamMixin type

  aybruhm

  committed
  5957b7b

• Update - added default value to BoolMeta instance

  aybruhm

  committed
  531fd3f

• Update - modified handle checkbox change function

  aybruhm

  committed
  ec6029e

• Feat - created llm example app for binaryparam

  aybruhm

  committed
  6c3197e

• Update - modified handle checkbox change function to fix lint error

  aybruhm

  committed
  9c73a5a

• Update - make use of 1/0 in BinaryParam default

  aybruhm

  committed
  632518f

• Add job extraction templates and tutorials

  

  mmabrouk

  authored and

  aybruhm

  committed
  cf4a604

• Add BinaryParam to config_datatypes.mdx

  

  mmabrouk

  authored and

  aybruhm

  committed
  cdeebf0

• Add Cookbook section to mint.json

  

  mmabrouk

  authored and

  aybruhm

  committed
  17afbf6

• Add async version of baby name generator app

  

  mmabrouk

  authored and

  aybruhm

  committed
  111f1ca

• Update job_info_extractor README and app.py

  

  mmabrouk

  authored and

  aybruhm

  committed
  50f67ba

• Update pyproject.toml version to 0.6.7 and update fastapi and ipdb dependencies

  

  mmabrouk

  authored and

  aybruhm

  committed
  3c64c78

• Update README.md

  

  mmabrouk

  authored and

  aybruhm

  committed
  4ee934c

• Update - modified BinaryParam sdk type

  aybruhm

  committed
  7f7e1d7

• Update - include boolean type in handleParamChange function

  aybruhm

  committed
  e365aa5

• Update - modified default parameter

  aybruhm

  committed
  204bf07

• Update README.md

  

  mmabrouk

  authored and

  aybruhm

  committed
  73e829b

• Update README.md

  

  mmabrouk

  authored and

  aybruhm

  committed
  eb8a9b7

• Update README.md

  

  mmabrouk

  authored and

  aybruhm

  committed
  c4e0be7

• Bump fastapi version to 0.95.1

  

  mmabrouk

  authored and

  aybruhm

  committed
  ccd5c0a

• fix: setting docker to always restart for both base-containers and apps

  

  romainrbr

  authored and

  aybruhm

  committed
  b04e5dc

• add restart always local dev docker compose

  

  aakrem

  authored and

  aybruhm

  committed
  1b4c028

• docs: update README.md [skip ci]

  

  allcontributors[bot]

  authored and

  aybruhm

  committed
  6f77e11

• docs: update .all-contributorsrc [skip ci]

  

  allcontributors[bot]

  authored and

  aybruhm

  committed
  3a7f3ec

• Add tutorials and templates for text generation and summarization tasks

  

  mmabrouk

  authored and

  aybruhm

  committed
  2589dd2

• Update title in deploy-mistral-model.mdx

  

  mmabrouk

  authored and

  aybruhm

  committed
  6ae371c

• Add RAG application tutorial and setup instructions

  

  mmabrouk

  authored and

  aybruhm

  committed
  c7a7895

• Add "build-rag-application" tutorial to mint.json

  

  mmabrouk

  authored and

  aybruhm

  committed
  8d8a8d6

• Add RAG application tutorial with LlamaIndex

  

  mmabrouk

  authored and

  aybruhm

  committed
  e0713f0

• Add tut_llama_index_1.png image to tutorial-rag-application

  

  mmabrouk

  authored and

  aybruhm

  committed
  a2888db

• added rag tutorial

  

  mmabrouk

  authored and

  aybruhm

  committed
  55890df

• Update tutorial with link to full code

  

  mmabrouk

  authored and

  aybruhm

  committed
  71f5aad

• fix BaseSettings PydanticImportError

  

  devgenix

  authored and

  aybruhm

  committed
  b735d21

• fix / causing delete variant to fail in cloud

  

  devgenix

  authored and

  aybruhm

  committed
  e2ff04d

• fix pydantic

  

  devgenix

  authored and

  aybruhm

  committed
  77be6e5

• define function for openapi in cloud

  

  devgenix

  authored and

  aybruhm

  committed
  7bbc7f0

• regenerate backend and set timeout to 60

  

  devgenix

  authored and

  aybruhm

  committed
  479c5b2

• format

  

  devgenix

  authored and

  aybruhm

  committed
  bbc9a99

• set timeout to 600 for build_image function

  

  devgenix

  authored and

  aybruhm

  committed
  d7e6d67

• add step to set timeout

  

  devgenix

  authored and

  aybruhm

  committed
  1aeec53

• Update Readme.md with image description

  

  devgenix

  authored and

  aybruhm

  committed
  78ba872

Commits on Jan 4, 2024

• Merge pull request #1096 from Agenta-AI/Sub-issue-1059/-Add-cost,-latency,-tokens-to-playground

  Show description for 01ba0d1
  mmabrouk

  authored
  01ba0d1

• Merge branch 'main' into feature/add-metrics-to-sdk-playground-evaluation

  mmabrouk

  committed
  b3eed50

• Merge pull request #1111 from Agenta-AI/gh/update_templates_cost_latency

  Show description for f29983d
  mmabrouk

  authored
  f29983d

• Update main.py

  mmabrouk

  authored
  a294424

• Fix result type conversion in evaluation tables

  mmabrouk

  committed
  abf1ec1

• remove cost, latency and usage if res is a string

  bekossy

  committed
  fdf797c

• null checks added

  bekossy

  committed
  e8caf7e

Commits on Jan 5, 2024

• Merge pull request #1155 from Agenta-AI/bugfix/evaluation_new_sdk

  Show description for e9b5fad
  mmabrouk

  authored
  e9b5fad

Commits on Jan 8, 2024

• fix broken links

  devgenix

  committed
  10f7579

• register_default now per default does not overwrite

  mmabrouk

  committed
  481d431

• allow overwrite when parameters are empty even if overwrite is set to False

  mmabrouk

  committed
  a5d3549

• Update variant image now removes configuration

  mmabrouk

  committed
  af5b33f

• formatting

  mmabrouk

  committed
  a9c2d35

• Fix

  mmabrouk

  committed
  396eee4

• Update link to LLM app tutorial

  mmabrouk

  committed
  5b7ad6e

• Update tutorial and template links

  mmabrouk

  committed
  16d8c1b

• Update links in deprecated documentation

  mmabrouk

  committed
  b23c990

• Update links in documentation

  mmabrouk

  committed
  4549925

• Update links in README.md

  mmabrouk

  committed
  72f47e8

• Merge pull request #1151 from Agenta-AI/feature/add-metrics-to-sdk-playground-evaluation

  Show description for 0636c5a
  mmabrouk

  authored
  0636c5a

• Merge pull request #1166 from Agenta-AI/1147-bug-fix-deadlinks-in-the-documentation

  Show description for 693fbdc
  mmabrouk

  authored
  693fbdc

• Merge pull request #1169 from Agenta-AI/bugfix/saving_parameters

  Show description for 3aa948a
  mmabrouk

  authored
  3aa948a

Commits on Jan 10, 2024

• Update pyproject.toml

  mmabrouk

  authored
  0e3957e

• Merge pull request #1180 from Agenta-AI/mmabrouk-patch-4

  Show description for 65081e2
  mmabrouk

  authored
  65081e2

Commits on Jan 11, 2024

• hotfix

  mmabrouk

  committed
  086bb76

• Merge pull request #1183 from Agenta-AI/hotfix/pydantic_downgrade

  Show description for 044130a
  mmabrouk

  authored
  044130a

• Merge branch 'main' into main-to-evaluations-in-backend

  aakrem

  committed
  89c355e