Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Observability AI Assistant] fix: Improve tool choice handling in Observability AI Assistant client #203928

Conversation

arturoliduena
Copy link
Contributor

@arturoliduena arturoliduena commented Dec 11, 2024

Closes #203926
Error:

ai_assistant_broken.mov

logs:

[2024-12-11T11:03:19.666-05:00][WARN ][plugins.actions.gen-ai] action execution failure: .gen-ai:azure-gpt4: GPT-4 Azure: an error occurred while running the action: Status code: 400. Message: API Error: #model_error - [] should be non-empty - 'tools'; retry: true
[2024-12-11T11:03:19.669-05:00][ERROR][plugins.observabilityAIAssistant.service] Error: Unexpected error
    at createInferenceInternalError (/Users/viduni/Workspace/Elastic/kibana/x-pack/packages/ai-infra/inference-common/src/errors.ts:49:10

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

@arturoliduena arturoliduena added release_note:fix backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Team:Obs AI Assistant Observability AI Assistant backport:version Backport to applied version labels v8.18.0 labels Dec 11, 2024
@arturoliduena arturoliduena requested a review from a team as a code owner December 11, 2024 20:34
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Dec 11, 2024
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

Copy link
Contributor

@viduni94 viduni94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Added a couple of comments (nitpicks).

@arturoliduena arturoliduena removed the backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) label Dec 12, 2024
@arturoliduena arturoliduena force-pushed the 203926-Obs-AI-Assistant-Error-model_error-tools-should-benon-empty branch from ee1ca19 to fd69535 Compare December 12, 2024 11:57
@arturoliduena arturoliduena changed the title fix: Improve tool choice handling in Observability AI Assistant client [Observability AI Assistant] fix: Improve tool choice handling in Observability AI Assistant client Dec 12, 2024
@elasticmachine
Copy link
Contributor

elasticmachine commented Dec 12, 2024

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: fd69535
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-203928-fd69535aa3d6

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #28 / aiops change point detection UI allows change point selection for detailed view

Metrics [docs]

✅ unchanged

History

cc @arturoliduena

@arturoliduena arturoliduena merged commit 4455087 into elastic:main Dec 12, 2024
8 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12297905874

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 12, 2024
…ervability AI Assistant client (elastic#203928)

Error:

https://github.com/user-attachments/assets/3ada87df-9c60-4916-aa5a-7f95fb2dc275

logs:
``` bash
[2024-12-11T11:03:19.666-05:00][WARN ][plugins.actions.gen-ai] action execution failure: .gen-ai:azure-gpt4: GPT-4 Azure: an error occurred while running the action: Status code: 400. Message: API Error: #model_error - [] should be non-empty - 'tools'; retry: true
[2024-12-11T11:03:19.669-05:00][ERROR][plugins.observabilityAIAssistant.service] Error: Unexpected error
    at createInferenceInternalError (/Users/viduni/Workspace/Elastic/kibana/x-pack/packages/ai-infra/inference-common/src/errors.ts:49:10
```

---------

Co-authored-by: Viduni Wickramarachchi <[email protected]>
(cherry picked from commit 4455087)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@arturoliduena
Copy link
Contributor Author

Closes #203926

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:version Backport to applied version labels ci:project-deploy-observability Create an Observability project release_note:fix Team:Obs AI Assistant Observability AI Assistant v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Obs AI Assistant] Error: model_error - [] should be non-empty - 'tools'
4 participants