Code

Feat - custom code evaluation in UI#589

Code

Closed

aybruhm wants to merge 90 commits intoAgenta-AI:mainAgenta-AI/agenta:mainfrom aybruhm:gh/custom-code-evaluation-in-uiaybruhm/agenta:gh/custom-code-evaluation-in-uiCopy head branch name to clipboard

+3,293-916

Commits

Commits on Sep 10, 2023

• Update - added restrictedpython

  aybruhm

  committed
  d5774bf

• Feat - created security module

  aybruhm

  committed
  e2d57bd

• Feat - implemented execute_code_safely function

  aybruhm

  committed
  3cefed1

Commits on Sep 11, 2023

• Feat - created custom evaluation db collection

  aybruhm

  committed
  7e74d5a

• Feat - created custom evaluation type and store custom evaluation api models

  aybruhm

  committed
  3e0ee16

• Feat - implemented store and execute custom code evaluation logics

  aybruhm

  committed
  ad4fe0d

• Feat - implemented function to check if module import is safe to ensure code safe execution

  aybruhm

  committed
  f4bd4ea

• Cleanup - remove app_name from execute_custom_code_evaluation

  aybruhm

  committed
  05107f1

• Feat - implemented store and execute custom evaluation routers

  aybruhm

  committed
  182c812

• Update - added custom_code_run to evaluation type and labels

  aybruhm

  committed
  7e8e805

• Feat - upload custom_code image

  aybruhm

  committed
  1e6c7d2

• Feat - created store custom evaluation type interface

  aybruhm

  committed
  b9c705d

• Feat - created type interface for store custom evaluation success response

  aybruhm

  committed
  45e3f0d

• Feat - implemented save custom code evaluation api logic

  aybruhm

  committed
  37ab07f

• Feat - implemented custom evaluation dropdown component

  aybruhm

  committed
  db97338

• Update - added type dropdown component

  aybruhm

  committed
  296b461

• Feat - implemented custom python code component

  aybruhm

  committed
  e97c9f6

• Refactor - renamed component prop interface

  aybruhm

  committed
  9219fb7

• Feat - created type interface for single custom evaluation

  aybruhm

  committed
  998a0af

• Feat - implemented axios logic to fetch custom evaluations

  aybruhm

  committed
  9949093

• Update - improve security in sandbox environment

  aybruhm

  committed
  7cc58a8

• Cleanup - removed custom evaluation type embedded model and some fields in custom evaluation db

  aybruhm

  committed
  acdd892

• Feat - implemented fetch custom evaluations evaluation service

  aybruhm

  committed
  e4e3fa2

• Feat - implemented list custom evaluations api router

  aybruhm

  committed
  193f619

• Feat - created custom evaluation output and added new type in evaluation type schema

  aybruhm

  committed
  7b9987a

Commits on Sep 12, 2023

• Update - modified custom evaluations dropdown component to set custom evaluation id

  aybruhm

  committed
  7619a9b

• Update - include custom python code and evaluation dropdowns component in evaluations component

  aybruhm

  committed
  0b79b68

• Refactor - removed custom_code.png

  aybruhm

  committed
  5670435

• Feat - created evaluation api model to execute custom evaluation code

  aybruhm

  committed
  7e708d7

• Feat - implemented custom code run evaluation page

  aybruhm

  committed
  3b1a5b6

• Feat - implemented helper function to include dynamic values

  aybruhm

  committed
  e23e92a

• Update - add condition to save correct_answer for cusutom_code evaluations

  aybruhm

  committed
  b008773

• Feat - created type interface for execute custom eval code

  aybruhm

  committed
  459f400

• Feat - implemented axios logic to execute custom evaluation code

  aybruhm

  committed
  dd13b63

• Update - added optional field (correct_answer)

  aybruhm

  committed
  530f72d

• Feat - implemented fetch average score for custom code run result service

  aybruhm

  committed
  3b10006

• Update - modified fetch_results and execute_custom_evaluation routers

  aybruhm

  committed
  9d7a87c

• Cleanup - remove unused code-blocks

  aybruhm

  committed
  5ba12ea

• Feat - implemented custom code run evaluation table component

  aybruhm

  committed
  06ee96f

• 🎨 Format - ran format-fix and black

  aybruhm

  committed
  e2aa26a

• Merge branch 'main' into gh/custom-code-evaluation-in-ui

  aybruhm

  authored
  ed19a31

• Feat - created create custom evaluation page

  aybruhm

  committed
  67972ea

• Update - removed custom python code in evaluation component

  aybruhm

  committed
  5f7c7b3

• Cleanup - formatted custom evaluations dropdown component

  aybruhm

  committed
  a80fdac

• Refactor - renamed saveCutomCodeEvaluation to saveCustomCodeEvaluation

  aybruhm

  committed
  772635b

• Update - added new styles

  aybruhm

  committed
  6422f06

• Update - introduce pre-filled example of an evaluation function and syntax-highling on the example code

  aybruhm

  committed
  ba348fc

• 🎨 Format - ran format-fix and black

  aybruhm

  committed
  2d9e359

• Update - added variant_name to type interface ExecuteCustomEvalCode

  aybruhm

  committed
  4c2a6e1

• Update - added app_params, output to sandbox and allow execute of evaluation function

  aybruhm

  committed
  82aaa8c

• Update - added output to execute_custom_code_execution service function

  aybruhm

  committed
  d47950b

• Update - added app_name, variant_name, and outputs to execute_custom_evaluation_code api model

  aybruhm

  committed
  4eae667

• Refactor - modified executeCustomEvaluationCode axios api logic

  aybruhm

  committed
  4b3e778

• Update - added styles for copy btn in custom python code component

  aybruhm

  committed
  6c6bb84

• Update - refactor evaluate function and added new args in callCUstomCodeHandler function

  aybruhm

  committed
  70a7a0b

• Update - add custom code evaluation id to evaluation

  aybruhm

  committed
  c3e14ed

• Update - retrieve evaluations for custom code evals

  aybruhm

  committed
  6075cf1

• Update -added custom code evalation id to router push

  aybruhm

  committed
  71736a9

• Update - added custom_code_evaluation_id and made it optional

  aybruhm

  committed
  3df9720

• Update - added btn to copy code example for custom evaluation function

  aybruhm

  committed
  25b9715

• Update - created format_outputs helper function

  aybruhm

  committed
  1b0e4b7

• Update - added correct_answer to execute custom evaluation code api model

  aybruhm

  committed
  0905050

• Update - modified evaluation function example description

  aybruhm

  committed
  087160c

• Update - modified fetch_average_score_for_custom_code_run

  aybruhm

  committed
  4ae2c74

• Feat - created update_evaluation_scenario_score logic and added doc strings to custom evaluations services

  aybruhm

  committed
  17235cd

• Update - include correct_answer to custom eval code params

  aybruhm

  committed
  ef95da8

• Feat - implemented update evaluation scenario score axios logic

  aybruhm

  committed
  ded6649

• Feat - created evaluation scenario score update api model

  aybruhm

  committed
  d7f2584

• Update - receive put data by payload instead of query

  aybruhm

  committed
  d2ae93a

• Update - modified custom code run evaluation table component

  aybruhm

  committed
  b6a80b6

• 🎨 Format - ran format-fix and black

  aybruhm

  committed
  175ef80

• Update - installed packages.json

  aybruhm

  committed
  66fb647

• Update - integrated ace editor for code input and syntax highlighting

  aybruhm

  committed
  d4a33f7

• Update - set result and avg_score to 2 decimal places

  aybruhm

  committed
  c01e81e

• 🎨 Format - ran format-fix

  aybruhm

  committed
  f9b8929

Commits on Sep 13, 2023

• Cleanup - add ? to handle undefined error

  aybruhm

  committed
  253777b

• 🎨 Format - ran format-fix

  aybruhm

  committed
  3a98f9f

• Merge branch 'main' into gh/custom-code-evaluation-in-ui

  aybruhm

  committed
  b05fa01

• 🎨 Format - ran format-fix and black

  aybruhm

  committed
  7ef2d6a

• Cleanup - removed raise exception when no custom evaluations is found

  aybruhm

  committed
  6d74e30

Commits on Sep 14, 2023

• Refactor - override error interceptor for get all variant parameters api call

  aybruhm

  committed
  67442cb

• Cleanup - removed console log

  aybruhm

  committed
  ec1452d

• Feat - created backend router to get evaluation scenario score and axios logic to make backend call

  aybruhm

  committed
  1160122

• Update - round score by 2 decimal

  aybruhm

  committed
  df09373

• Refactor - removed CustomEvaluationsDropdown component

  aybruhm

  committed
  a6e908b

• Refactor - improve get_evaluation_scenario_score_router

  aybruhm

  committed
  493d166

• Refactor - directly include dropdown select of custom evaluations

  aybruhm

  committed
  e498683

• Update - added logic to fetch results of ran evaluation scenarios

  aybruhm

  committed
  89b579e

• 🎨 Format - ran format-fix and black

  aybruhm

  committed
  b93c371

• Cleanup - fix type error

  aybruhm

  committed
  3c5fcfa