Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: unclear system output when unable to find answr #90

Open
sfc-gh-alherrera opened this issue Dec 11, 2024 · 1 comment
Open

BUG: unclear system output when unable to find answr #90

sfc-gh-alherrera opened this issue Dec 11, 2024 · 1 comment

Comments

@sfc-gh-alherrera
Copy link
Collaborator

introduced with the dynamic output update. If answer not found, the replanning message is used for dynamic output. we need to append some pre-canned language to the dynamic output for it to make more sense.

agent("What is market cap of apple")

if analyst tool fails for whatever reason and it can't retrieve the answer, user would see "what is the market cap of apple" with the curent implementation

@sfc-gh-twhite
Copy link
Collaborator

Here is a full log dump (reduced lists and ids for brevity) with a slightly derived Quickstart notebook execution.

annual_reports = CortexSearchTool(**search_config)
sp500 = CortexAnalystTool(**analyst_config)

snowflake_tools = [annual_reports, sp500]
agent = Agent(snowflake_connection=snowpark, tools=snowflake_tools)
agent("Give me recent news about X")
INFO:AgentGatewayLogger:Cortex Search Tool successfully initialized
INFO:AgentGatewayLogger:Cortex Analyst Tool successfully initialized
INFO:AgentGatewayLogger:Cortex gateway successfully initialized
DEBUG:AgentGatewayLogger:LLM Generated Plan:
 Thought: I need to search for recent news about X.
1. sec_search_service_cortexsearch("Recent news about X")
Thought: I can answer the question now.
2. fuse()<END_OF_PLAN>
INFO:AgentGatewayLogger:running sec_search_service_cortexsearch task
DEBUG:AgentGatewayLogger:Cortex Search Query:Recent news about X
DEBUG:AgentGatewayLogger:Cortex Search Response:{'results': [...], 'request_id': '318ba9e3...'}
DEBUG:AgentGatewayLogger:task successfully completed
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:Question: 

DEBUG:AgentGatewayLogger:Give me recent news about X
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:Raw Answer: 

DEBUG:AgentGatewayLogger: Thought: The observations do not provide recent news about X.
Action: Replan(Give me recent news about X)
<END_OF_RESPONSE>
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:LLM Generated Plan:
 Thought: The previous plan searched for recent news about X but did not find relevant information. I need to search for recent news about X again.

1. sec_search_service_cortexsearch("Recent news about X")

Thought: I can answer the question now.

2. fuse()

<END_OF_PLAN>
INFO:AgentGatewayLogger:running sec_search_service_cortexsearch task
DEBUG:AgentGatewayLogger:Cortex Search Query:Recent news about X
DEBUG:AgentGatewayLogger:Cortex Search Response:{'results': [...], 'request_id': 'bb7ee765...'}
DEBUG:AgentGatewayLogger:task successfully completed
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:Question: 

DEBUG:AgentGatewayLogger:Give me recent news about X
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:================================================================================
DEBUG:AgentGatewayLogger:Raw Answer: 

DEBUG:AgentGatewayLogger: Thought: The observations do not provide recent news about X. I need to search for more relevant information.

Action: Replan(Give me recent news about X)

<END_OF_RESPONSE>
DEBUG:AgentGatewayLogger:================================================================================
"Give me recent news about X"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants