DuckDuckGo RateLimit Exception on running example/SearchManager.py

[VigneshHexo]

While running the SearchManager example, rate limit exception arises despite of running it the very first time as well. I think the Agent should be programmed to handle this exception as well. Please let me know if you could fix this? Thank you very much

I would be happy to fix it as well and create a PR if you think this could be helpful. Here is the error trace for your reference

python example/SearchManager.py
Agent Search_Manager receives the following TaskPackage:
[
        Task ID: c07e4633-5147-481e-8a64-8bbc736300b4
        Instruction: what is micorsoft famous for
]
====Search_Manager starts execution on TaskPackage c07e4633-5147-481e-8a64-8bbc736300b4
====                                                                                   /opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_core/_api/deprecation.
py:117: LangChainDeprecationWarning: The function `run` was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use invoke instead.                                     warn_deprecated(
Agent Search_Manager takes 0-step Action:
{
        name: Think
        params: {'response': "I should first ask duck_search_agent to search for inform
ation about Microsoft, if I don't find an answer, I will ask wiki_search_agent to search for information about Microsoft"}                                                    }
Observation: OK
Agent Search_Manager takes 1-step Action:
{
        name: duck_search_agent
        params: {'Task': 'search the information about Microsoft'}
}
Agent duck_search_agent receives the following TaskPackage:
[
        Task ID: c07e4633-5147-481e-8a64-8bbc736300b4
        Instruction: search the information about Microsoft
]
====duck_search_agent starts execution on TaskPackage c07e4633-5147-481e-8a64-8bbc73630
0b4====                                                                                Agent duck_search_agent takes 0-step Action:
{
        name: DuckDuckGo_Search
        params: {'query': 'Microsoft information'}
}
Traceback (most recent call last):
  File "/home/ubuntu/AgentLite/example/SearchManager.py", line 82, in <module>
    test_manager_agent()
  File "/home/ubuntu/AgentLite/example/SearchManager.py", line 77, in test_manager_agen
t                                                                                          response = search_manager(test_task_pack)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/AgentLite/agentlite/agents/BaseAgent.py", line 115, in __call__
    self.execute(task)
  File "/home/ubuntu/AgentLite/agentlite/agents/BaseAgent.py", line 150, in execute
    observation = self.forward(task, action)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/AgentLite/agentlite/agents/ManagerAgent.py", line 157, in forward
    observation = agent(new_task_package)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/AgentLite/agentlite/agents/BaseAgent.py", line 115, in __call__
    self.execute(task)
  File "/home/ubuntu/AgentLite/agentlite/agents/BaseAgent.py", line 150, in execute
    observation = self.forward(task, action)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/AgentLite/agentlite/agents/BaseAgent.py", line 238, in forward
    observation = action(**agent_act.params)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/AgentLite/example/SearchActions.py", line 20, in __call__
    return self.search.run(query)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_core/tools.py"
, line 401, in run                                                                         raise e
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_core/tools.py"
, line 358, in run                                                                         self._run(*tool_args, run_manager=run_manager, **tool_kwargs)
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_community/tool
s/ddg_search/tool.py", line 64, in _run                                                    res = self.api_wrapper.results(query, self.max_results, source=self.backend)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_community/util
ities/duckduckgo_search.py", line 111, in results                                          for r in self._ddgs_text(query, max_results=max_results)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/langchain_community/util
ities/duckduckgo_search.py", line 48, in _ddgs_text                                        ddgs_gen = ddgs.text(
               ^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search.py", line 57, in text                                                          return self._run_async_in_thread(super().text(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search.py", line 53, in _run_async_in_thread                                          result = future.result()
             ^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/concurrent/futures/_base.py", line 456
, in result                                                                                return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/agentlite/lib/python3.11/concurrent/futures/_base.py", line 401
, in __get_result                                                                          raise self._exception
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search_async.py", line 167, in text                                                   results = await self._text_api(keywords, region, safesearch, timelimit, max_results
)                                                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^                                                                                        File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search_async.py", line 258, in _text_api                                              raise e
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search_async.py", line 253, in _text_api                                              await asyncio.gather(*tasks)
  File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search_async.py", line 229, in _text_api_page                                         resp_content = await self._aget_url("GET", "https://links.duckduckgo.com/d.js", par
ams=payload)                                                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^                                                                             File "/opt/conda/envs/agentlite/lib/python3.11/site-packages/duckduckgo_search/duckdu
ckgo_search_async.py", line 124, in _aget_url                                              raise RatelimitException(f"{resp.url} {resp.status_code} Ratelimit")
duckduckgo_search.exceptions.RatelimitException: https://links.duckduckgo.com/d.js?q=Mi
crosoft+information&kl=wt-wt&l=wt-wt&p=&s=0&df=y&vqd=4-7297422369529734716423891034146014912&ex=-1 202 Ratelimit                                                              

[JimSalesforce]

Yes, we originally use the duckduckgo-search from langchain tools. But it is already deprecated. We updated the new example with update-to-date tools.

[JimSalesforce]

#20

[VigneshHexo]

Thanks Jim! Will retry the new example and keep you posted