Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement]: observability integration without using agenta hosted apps #1567

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
c90fcfc
Refactor: added tracing when running llm app on terminal
aybruhm Apr 26, 2024
447282b
Refactor: implemented sdkclient class to build sdk clien instance and…
aybruhm Apr 26, 2024
88b88d8
Refactor: rewrite app_async example to allow tracing to work on termi…
aybruhm Apr 26, 2024
67d0f72
Style: format agenta_decorator with black
aybruhm Apr 26, 2024
cd18d5e
Build: tag sdk version to 0.13.7a0
aybruhm Apr 26, 2024
5d937d9
Docs: update tracing docstrings and set variant_id to be optional
aybruhm May 1, 2024
af71a30
Feat: created base decorator class
aybruhm May 2, 2024
4b15ef5
Refactor: consolidate decorators into new decorators folder, remove a…
aybruhm May 2, 2024
64aa24e
Refactor: update imports for new decorators
aybruhm May 2, 2024
e8bef98
Refactor: update app_async llm app example to show use of new entrypo…
aybruhm May 2, 2024
6d460eb
Refactor: improved singleton design and condition to enable/disable l…
aybruhm May 2, 2024
869dd24
Style: format with black
aybruhm May 2, 2024
688e63b
Merge branch 'main' into 1560-age-118-observability-integration-witho…
aybruhm May 2, 2024
4c9469b
SDK: bump version to 0.14.1a1 for pre-release
aybruhm May 2, 2024
79fa25f
Minor Refactor: added docstrings to entrypoint decorator class
aybruhm May 2, 2024
83cb75f
Refactor: remove duplicate use of span kind in result error login whe…
aybruhm May 9, 2024
c1e946a
Refactor: set parent_span config default to be an empty directory and…
aybruhm May 9, 2024
390ec5c
Feat: created trace decorator for observability
aybruhm May 9, 2024
5a2ad88
Refactor: clean up llm_entrypoint to not have any observability logic
aybruhm May 9, 2024
3750194
Refactor: remove use of redundant SDKClient class and replace with Ag…
aybruhm May 9, 2024
fa1464a
Refactor: import trace and Tracing in __init__ modules
aybruhm May 9, 2024
803b35e
Feat: include workflows based on observability integration without ag…
aybruhm May 9, 2024
16a2ebd
Style: format agenta-cli/ with black
aybruhm May 9, 2024
525c725
Cleanup: remove flag to enable tracing in Tracing SDK
aybruhm May 9, 2024
492c1bb
Merge 'main' into 'origin/1560-age-118-observability-integration-with…
aybruhm May 9, 2024
6a17d02
SDK: bump version to 0.14.6a0 for internal test
aybruhm May 9, 2024
4385f07
Refactor: added check to raise value error if order of trace and entr…
aybruhm May 12, 2024
57fe30e
Refactor: observe llm func with trace decorator when enable_tracing i…
aybruhm May 13, 2024
c8997a7
Refactor: rename *_PLAYGROUND to *_ENVIRONMENT and improve tracing_pl…
aybruhm May 13, 2024
4a756eb
Minor refactor: update entrypoint class decorator docstring
aybruhm May 13, 2024
fcd7d01
Merge 'main' branch into '1560-age-118-observability-integration-with…
aybruhm May 13, 2024
67fc1c0
SDK: bump pre-release version to 0.14.7a0
aybruhm May 13, 2024
8a6759f
Minor refactor (llm_entrypoint): remove __init__ method and condition…
aybruhm May 23, 2024
fb244b6
Refactor (tracing): merge trace and span decorator into instrument an…
aybruhm May 23, 2024
6a4ccb2
Refactor (agenta_init): revert agenta singleton back to initial state
aybruhm May 23, 2024
ba33217
Refactor (llm_tracing): simplify flow of tracing
aybruhm May 23, 2024
7493060
Refactor (tracing): initialize Tracing sdk in instrument decorator
aybruhm May 23, 2024
fc444d4
Refactor (workflows): update usecases of using observability with agenta
aybruhm May 23, 2024
411cb21
Style: format code with black
aybruhm May 23, 2024
4f39e4a
Merge branch 'main' into 1560-age-118-observability-integration-witho…
aybruhm May 23, 2024
6645123
Style: reformat code with [email protected]
aybruhm May 23, 2024
7ffd6a2
SDK: bump pre-release version to 0.14.12a0
aybruhm May 23, 2024
2ceae7f
Refactor (tracing): set global variable to set span attribute
aybruhm May 23, 2024
272d4ab
Build: swap pip installations
aybruhm May 23, 2024
e94d698
Minor refactor (workflow): update use of set_span_attribute
aybruhm May 23, 2024
fbf0059
SDK: bump pre-release version to 0.14.12a1
aybruhm May 23, 2024
1a5c40a
Minor refactor: fixed 'SDKClient' is not defined
aybruhm May 23, 2024
708d670
SDK: bump pre-release version to 0.14.12a2
aybruhm May 23, 2024
920637b
Minor refactor: fix failing serve and update is_main_script logic
aybruhm May 23, 2024
e89febc
Minor refactor: cleanup examples app for observability
aybruhm May 28, 2024
48e92ba
Minor refactor: allow initialization of tracing sdk in agenta_init
aybruhm May 28, 2024
0ef3913
Refactor: set the configuration/environment of the LLM app at run-tim…
aybruhm May 28, 2024
71915d4
Refactor (llm_tracing): maintain naming consistency with host and imp…
aybruhm May 28, 2024
5d71490
Style: format code with [email protected]
aybruhm May 28, 2024
9b76c72
Refactor (instrument): improve decorator to handle async and sync fun…
aybruhm May 28, 2024
120fe86
updatee tracing class
mmabrouk May 29, 2024
f7c236b
update decorator llm endpoint
mmabrouk May 29, 2024
5749f0f
update_examples
mmabrouk May 29, 2024
bf561e1
fixes
mmabrouk May 29, 2024
754b4d8
Merge pull request #1716 from Agenta-AI/fix-obs-pr
aybruhm May 29, 2024
6da1811
Update pyproject.toml
mmabrouk May 29, 2024
683037d
Merge branch 'main' into 1560-age-118-observability-integration-witho…
aybruhm May 29, 2024
7cf2f03
Style: format code with [email protected]
aybruhm May 29, 2024
5f52761
Update Dockerfile.cloud.template
mmabrouk May 29, 2024
b131f72
pyroject
mmabrouk May 29, 2024
3d0cf10
Refactor (agenta_init): update use of backend api client
aybruhm May 29, 2024
cce47f3
SDK: bump pre-release version to 0.15.0a3
aybruhm May 30, 2024
dcf4804
removed variant_id from the tracing initialization
mmabrouk May 30, 2024
b765759
- Fixed client initialization
mmabrouk May 30, 2024
6e6e534
addded the option of using config file for initialization
mmabrouk May 30, 2024
1caae8e
improved tracing to trace inputs and outputs for all spans
mmabrouk May 30, 2024
569d016
format
mmabrouk May 30, 2024
9464b7d
fix tests
mmabrouk May 30, 2024
52fdafc
Fix pickle issue
mmabrouk May 30, 2024
cc8d287
format
mmabrouk May 30, 2024
683ab9a
Bug Fix (llm_entrypoint): update is_main_script logic
aybruhm May 30, 2024
514f2dc
SDK: bump pre-release to 0.15.0a4
aybruhm May 30, 2024
ce92c6a
fixed entry point
mmabrouk May 30, 2024
046f2d7
improved value error
mmabrouk May 30, 2024
23bfaf0
removed need for () for entrypoint
mmabrouk May 30, 2024
9c0a667
modified the examples to remove paranthesis
mmabrouk May 30, 2024
0ca9392
Merge branch '1560-age-118-observability-integration-without-using-ag…
mmabrouk May 30, 2024
a1cef51
set logger to debug
mmabrouk May 30, 2024
80463b3
revert debug to false in cli
mmabrouk May 30, 2024
cdf696e
Merge branch 'fix-obs-pr-2' of https://github.com/Agenta-AI/agenta-co…
mmabrouk May 30, 2024
1cf5dfb
Update agenta_init.py
mmabrouk May 30, 2024
fe94f90
Update app_async.py
mmabrouk May 30, 2024
5d09dae
Update app_nested_async.py
mmabrouk May 30, 2024
9cb1326
Update tracing_plus_entrypoint.py
mmabrouk May 30, 2024
623197e
fix
mmabrouk May 30, 2024
46539a2
Merge pull request #1722 from Agenta-AI/fix-obs-pr-2
aybruhm May 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions agenta-backend/agenta_backend/services/app_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,6 @@ async def start_variant(
env_vars = {} if env_vars is None else env_vars
env_vars.update(
{
"AGENTA_VARIANT_NAME": db_app_variant.variant_name,
"AGENTA_VARIANT_ID": str(db_app_variant.id),
"AGENTA_BASE_ID": str(db_app_variant.base.id),
"AGENTA_APP_ID": str(db_app_variant.app.id),
"AGENTA_HOST": domain_name,
Expand Down
9 changes: 6 additions & 3 deletions agenta-cli/agenta/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
from .sdk.utils.preinit import PreInitObject
from .sdk.agenta_decorator import app, entrypoint
from .sdk.context import get_contexts, save_context
from .sdk.types import (
Context,
Expand All @@ -14,9 +13,13 @@
FileInputURL,
BinaryParam,
)
from .sdk.tracing.decorators import span
from .sdk.agenta_init import Config, init, llm_tracing
from .sdk.tracing.llm_tracing import Tracing
from .sdk.decorators.tracing import instrument
from .sdk.decorators.llm_entrypoint import entrypoint, app
from .sdk.agenta_init import Config, AgentaSingleton, init
from .sdk.utils.helper.openai_cost import calculate_token_usage
from .sdk.client import Agenta

config = PreInitObject("agenta.config", Config)
DEFAULT_AGENTA_SINGLETON_INSTANCE = AgentaSingleton()
tracing = DEFAULT_AGENTA_SINGLETON_INSTANCE.tracing # type: ignore
14 changes: 8 additions & 6 deletions agenta-cli/agenta/client/backend/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ def __init__(
self._client_wrapper = SyncClientWrapper(
base_url=base_url,
api_key=api_key,
httpx_client=httpx.Client(timeout=timeout)
if httpx_client is None
else httpx_client,
httpx_client=(
httpx.Client(timeout=timeout) if httpx_client is None else httpx_client
),
)
self.observability = ObservabilityClient(client_wrapper=self._client_wrapper)
self.apps = AppsClient(client_wrapper=self._client_wrapper)
Expand Down Expand Up @@ -1037,9 +1037,11 @@ def __init__(
self._client_wrapper = AsyncClientWrapper(
base_url=base_url,
api_key=api_key,
httpx_client=httpx.AsyncClient(timeout=timeout)
if httpx_client is None
else httpx_client,
httpx_client=(
httpx.AsyncClient(timeout=timeout)
if httpx_client is None
else httpx_client
),
)
self.observability = AsyncObservabilityClient(
client_wrapper=self._client_wrapper
Expand Down
2 changes: 1 addition & 1 deletion agenta-cli/agenta/client/backend/types/create_span.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,6 @@ def dict(self, **kwargs: typing.Any) -> typing.Dict[str, typing.Any]:
return super().dict(**kwargs_with_defaults)

class Config:
frozen = True
frozen = False
smart_union = True
json_encoders = {dt.datetime: serialize_datetime}
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
FROM public.ecr.aws/s2t9a1r1/agentaai/lambda_templates_public:main

COPY requirements.txt ${LAMBDA_TASK_ROOT}
RUN pip install --no-cache-dir --disable-pip-version-check -r requirements.txt
RUN pip install --no-cache-dir --disable-pip-version-check mangum
RUN pip install --no-cache-dir --disable-pip-version-check -U agenta
RUN pip install --no-cache-dir --disable-pip-version-check -U -r requirements.txt
RUN pip install --no-cache-dir --disable-pip-version-check mangum
COPY . ${LAMBDA_TASK_ROOT}

CMD [ "lambda_function.handler" ]
2 changes: 1 addition & 1 deletion agenta-cli/agenta/docker/docker-assets/Dockerfile.template
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ WORKDIR /app

COPY . .

RUN pip install --no-cache-dir --disable-pip-version-check -r requirements.txt
RUN pip install --no-cache-dir --disable-pip-version-check -U agenta
RUN pip install --no-cache-dir --disable-pip-version-check -r requirements.txt

EXPOSE 80

Expand Down
10 changes: 6 additions & 4 deletions agenta-cli/agenta/sdk/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
from .utils.preinit import PreInitObject # always the first import!
from . import agenta_decorator, context, types, utils # noqa: F401
from .agenta_decorator import app, entrypoint
from .context import get_contexts, save_context
from .types import (
Context,
Expand All @@ -15,9 +13,13 @@
FileInputURL,
BinaryParam,
)
from .tracing.decorators import span
from .agenta_init import Config, init, llm_tracing
from .tracing.llm_tracing import Tracing
from .decorators.tracing import instrument
from .decorators.llm_entrypoint import entrypoint, app
from .agenta_init import Config, AgentaSingleton, init
from .utils.helper.openai_cost import calculate_token_usage


config = PreInitObject("agenta.config", Config)
DEFAULT_AGENTA_SINGLETON_INSTANCE = AgentaSingleton()
tracing = DEFAULT_AGENTA_SINGLETON_INSTANCE.tracing # type: ignore
Loading
Loading