Adding ArXiv tool spec (#464)

llama_hub/tools/arxiv/README.md

@@ -0,0 +1,26 @@
+# ArXiv Search Tool
+
+This tool connects to ArXiv and allows an Agent to search for recent papers and their summaries to retrieve recent information on mathematical and scientific information
+
+## Usage
+
+This tool has a more extensive example usage documented in a Jupyter notebook [here](https://github.com/emptycrown/llama-hub/tree/main/llama_hub/tools/notebooks/arxiv.ipynb)
+
+Here's an example usage of the ArxivToolSpec.
+
+```python
+from llama_hub.tools.arxiv.base import ArxivToolSpec
+from llama_index.agent import OpenAIAgent
+
+tool_spec = ArxivToolSpec()
+
+agent = OpenAIAgent.from_tools(tool_spec.to_tool_list())
+
+agent.chat('Whats going on with the superconductor lk-99')
+agent.chat('what are the latest developments in machine learning')
+```
+
+`arxiv_query`: Search arXiv for results related to the query
+
+This loader is designed to be used as a way to load data as a Tool in a Agent. See [here](https://github.com/emptycrown/llama-hub/tree/main) for examples.
+

llama_hub/tools/arxiv/__init__.py

@@ -0,0 +1 @@
+## init

llama_hub/tools/arxiv/base.py

@@ -0,0 +1,34 @@
+"""arXiv tool spec."""
+
+from llama_index.tools.tool_spec.base import BaseToolSpec
+from typing import Optional
+from llama_index.readers.schema.base import Document
+
+class ArxivToolSpec(BaseToolSpec):
+    """arXiv tool spec."""
+
+    spec_functions = ["arxiv_query"]
+
+    def __init__(self, max_results: Optional[int] = 3):
+        self.max_results = max_results
+
+    def arxiv_query(self, query: str, sort_by: Optional[str] = "relevance"):
+        """
+        A tool to query arxiv.org
+        ArXiv contains a variety of papers that are useful for answering
+        mathematic and scientfic questions
+
+        Args:
+            query (str): The query to be passed to arXiv.
+            sort_by (str): Either 'relevance' (default) or 'recent'
+
+        """
+        import arxiv
+        sort = arxiv.SortCriterion.Relevance
+        if sort_by == 'recent':
+            sort = arxiv.SortCriterion.SubmittedDate
+        search =  arxiv.Search(query, max_results=self.max_results, sort_by=sort)
+        results = []
+        for result in search.results():
+            results.append(Document(text=f"{result.pdf_url}: {result.title}\n{result.summary}"))
+        return results

llama_hub/tools/arxiv/requirements.txt

@@ -0,0 +1 @@
+arxiv

llama_hub/tools/library.json

@@ -1,4 +1,9 @@
 {
+  "ArxivToolSpec": {
+    "id": "tools/arxiv",
+    "author": "ajhofmann",
+    "keywords": ["math", "science", "research"]
+  },
   "AzureCVToolSpec": {
     "id": "tools/azure_cv",
     "author": "ajhofmann",

llama_hub/tools/notebooks/arxiv.ipynb

@@ -0,0 +1,124 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "9a0ed198-6f5e-45dc-8a32-9273640dd563",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Setup OpenAI Agent\n",
+    "import openai\n",
+    "openai.api_key = 'sk-your-key'\n",
+    "from llama_index.agent import OpenAIAgent\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "0db1b7de-9392-44ec-b46b-9fa6994a37d1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "=== Calling Function ===\n",
+      "Calling function: arxiv_query with args: {\n",
+      "  \"query\": \"superconductor lk-99\"\n",
+      "}\n",
+      "Got output: [Document(id_='6a88492e-a18f-48e0-8f3a-55229c0a989b', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='c2874a35f19fb22be9f87f76be1c4d11d0d5fa4fada4f671fc97abd4a51ccfc6', text='http://arxiv.org/pdf/2307.16402v1: Synthesis of possible room temperature superconductor LK-99:Pb$_9$Cu(PO$_4$)$_6$O\\nThe quest for room-temperature superconductors has been teasing scientists\\nand physicists, since its inception in 1911 itself. Several assertions have\\nalready been made about room temperature superconductivity but were never\\nverified or reproduced across the labs. The cuprates were the earliest high\\ntransition temperature superconductors, and it seems that copper has done the\\nmagic once again. Last week, a Korean group synthesized a Lead Apatite-based\\ncompound LK-99, showing a T$_c$ of above 400$^\\\\circ$K. The signatures of\\nsuperconductivity in the compound are very promising, in terms of resistivity\\n(R = 0) and diamagnetism at T$_c$. Although, the heat capacity (C$_p$) did not\\nshow the obvious transition at T$_c$. Inspired by the interesting claims of\\nabove room temperature superconductivity in LK-99, in this article, we report\\nthe synthesis of polycrystalline samples of LK-99, by following the same heat\\ntreatment as reported in [1,2] by the two-step precursor method. The phase is\\nconfirmed through X-ray diffraction (XRD) measurements, performed after each\\nheat treatment. The room temperature diamagnetism is not evidenced by the\\nlevitation of a permanent magnet over the sample or vice versa. Further\\nmeasurements for the confirmation of bulk superconductivity on variously\\nsynthesized samples are underway. Our results on the present LK-99 sample,\\nbeing synthesized at 925$^\\\\circ$C, as of now do not approve the appearance of\\nbulk superconductivity at room temperature. Further studies with different heat\\ntreatments are though, yet underway.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), Document(id_='0aa54868-a98c-43d6-973f-198d86df1a66', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='c7c95ab3e761227447464e50d471dd7a95c09a4b703dd076b65d0c689ae63491', text='http://arxiv.org/pdf/1509.05138v1: Superfluid density of a pseudogapped superconductor near SIT\\nWe analyze critical behavior of superfluid density $\\\\rho_s$ in strongly\\ndisordered superconductors near superconductor-insulator transition and compare\\nit with the behavior of the spectral gap $\\\\Delta$ for collective excitations.\\nWe show that in contrast to conventional superconductors, the superconductors\\nwith preformed pairs display unusual scaling relation $\\\\rho_s \\\\propto \\\\Delta^2$\\nclose to superconductor-insulator transition. This relation have been reported\\nin very recent experiments.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), Document(id_='bb34a36a-703b-4ea6-9d49-5d95523b1d57', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='0ad1fad064b9ff0155aaf0b08ace70a4fb407386b0091442750a6517ac6faacf', text='http://arxiv.org/pdf/cond-mat/0405236v2: Supercurrent in Nodal Superconductors\\nIn recent years, a number of nodal superconductors have been identified;\\nd-wave superconductors in high T_c cuprates, CeCoIn$_5$, and\\n\\\\kappa-(ET)_2Cu(NCS)_2, 2D f-wave superconductor in Sr_2RuO_4 and hybrid\\ns+g-wave superconductor in YNi_2B_2C. In this work we conduct a theoretical\\nstudy of nodal superconductors in the presence of supercurrent. For simplicity,\\nwe limit ourselves to d-wave and 2D f-wave superconductors. We compute the\\nquasiparticle density of states and the temperature dependence of the depairing\\ncritical current in nodal superconductors, both of which are accessible\\nexperimentally.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n')]\n",
+      "========================\n",
+      "I found some information about the superconductor LK-99. A Korean group synthesized a Lead Apatite-based compound called LK-99, which showed a superconducting transition temperature (Tc) above 400°C. The compound exhibited promising signatures of superconductivity in terms of resistivity and diamagnetism at Tc. However, the heat capacity did not show an obvious transition at Tc. The synthesis of polycrystalline samples of LK-99 has been reported, but the bulk superconductivity at room temperature has not been confirmed yet. Further studies with different heat treatments are underway to investigate the superconducting properties of LK-99.\n",
+      "\n",
+      "Here are some papers related to the superconductor LK-99:\n",
+      "\n",
+      "1. [Synthesis of possible room temperature superconductor LK-99:Pb$_9$Cu(PO$_4$)$_6$O](http://arxiv.org/pdf/2307.16402v1)\n",
+      "2. [Superfluid density of a pseudogapped superconductor near SIT](http://arxiv.org/pdf/1509.05138v1)\n",
+      "3. [Supercurrent in Nodal Superconductors](http://arxiv.org/pdf/cond-mat/0405236v2)\n",
+      "\n",
+      "You can access these papers for more detailed information about LK-99 and its superconducting properties.\n"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_hub.tools.arxiv.base import ArxivToolSpec\n",
+    "\n",
+    "arxiv_tool = ArxivToolSpec()\n",
+    "\n",
+    "agent = OpenAIAgent.from_tools(\n",
+    "    arxiv_tool.to_tool_list(),\n",
+    "    verbose=True,\n",
+    ")\n",
+    "\n",
+    "print(agent.chat('Whats going on with the superconductor lk-99'))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "f519717e-c620-4e32-9729-99740b648556",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "=== Calling Function ===\n",
+      "Calling function: arxiv_query with args: {\n",
+      "  \"query\": \"machine learning\",\n",
+      "  \"sort_by\": \"recent\"\n",
+      "}\n",
+      "Got output: [Document(id_='9802fec1-89e5-447e-8134-ea79fff05d9f', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='66401a90439abe8b9f10e21217826da81e4a82909dbe4dee1833746dd50bf53f', text=\"http://arxiv.org/pdf/2308.06262v1: Foundation Model is Efficient Multimodal Multitask Model Selector\\nThis paper investigates an under-explored but important problem: given a\\ncollection of pre-trained neural networks, predicting their performance on each\\nmulti-modal task without fine-tuning them, such as image recognition,\\nreferring, captioning, visual question answering, and text question answering.\\nA brute-force approach is to finetune all models on all target datasets,\\nbringing high computational costs. Although recent-advanced approaches employed\\nlightweight metrics to measure models' transferability,they often depend\\nheavily on the prior knowledge of a single task, making them inapplicable in a\\nmulti-modal multi-task scenario. To tackle this issue, we propose an efficient\\nmulti-task model selector (EMMS), which employs large-scale foundation models\\nto transform diverse label formats such as categories, texts, and bounding\\nboxes of different downstream tasks into a unified noisy label embedding. EMMS\\ncan estimate a model's transferability through a simple weighted linear\\nregression, which can be efficiently solved by an alternating minimization\\nalgorithm with a convergence guarantee. Extensive experiments on 5 downstream\\ntasks with 24 datasets show that EMMS is fast, effective, and generic enough to\\nassess the transferability of pre-trained models, making it the first model\\nselection method in the multi-task scenario. For instance, compared with the\\nstate-of-the-art method LogME enhanced by our label embeddings, EMMS achieves\\n9.0\\\\%, 26.3\\\\%, 20.1\\\\%, 54.8\\\\%, 12.2\\\\% performance gain on image recognition,\\nreferring, captioning, visual question answering, and text question answering,\\nwhile bringing 5.13x, 6.29x, 3.59x, 6.19x, and 5.66x speedup in wall-clock\\ntime, respectively. The code is available at\\nhttps://github.com/OpenGVLab/Multitask-Model-Selector.\", start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), Document(id_='483086d9-6a3f-4ee2-9c99-d36afae56402', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='8ea334e93e68f69d5dc739feda3c5ff61079f8012f462b9b1a963809af9df14e', text='http://arxiv.org/pdf/2308.06261v1: Enhancing Network Management Using Code Generated by Large Language Models\\nAnalyzing network topologies and communication graphs plays a crucial role in\\ncontemporary network management. However, the absence of a cohesive approach\\nleads to a challenging learning curve, heightened errors, and inefficiencies.\\nIn this paper, we introduce a novel approach to facilitate a\\nnatural-language-based network management experience, utilizing large language\\nmodels (LLMs) to generate task-specific code from natural language queries.\\nThis method tackles the challenges of explainability, scalability, and privacy\\nby allowing network operators to inspect the generated code, eliminating the\\nneed to share network data with LLMs, and concentrating on application-specific\\nrequests combined with general program synthesis techniques. We design and\\nevaluate a prototype system using benchmark applications, showcasing high\\naccuracy, cost-effectiveness, and the potential for further enhancements using\\ncomplementary program synthesis techniques.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), Document(id_='b2d60846-255a-45d9-9f8a-4275845528be', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='7283a9c39ba44750d2c149f6d57c954c200e251665de0cdf5c64694eab9b11b3', text='http://arxiv.org/pdf/2308.06250v1: Big AI Models for 6G Wireless Networks: Opportunities, Challenges, and Research Directions\\nRecently, big artificial intelligence (AI) models represented by chatGPT have\\nbrought an incredible revolution. With the pre-trained big AI model (BAIM) in\\ncertain fields, numerous downstream tasks can be accomplished with only\\nfew-shot or even zero-shot learning and exhibit state-of-the-art performances.\\nAs widely envisioned, the big AI models are to rapidly penetrate into major\\nintelligent services and applications, and are able to run at low unit cost and\\nhigh flexibility. In 6G wireless networks, to fully enable intelligent\\ncommunication, sensing and computing, apart from providing other intelligent\\nwireless services and applications, it is of vital importance to design and\\ndeploy certain wireless BAIMs (wBAIMs). However, there still lacks\\ninvestigation on architecture design and system evaluation for wBAIM. In this\\npaper, we provide a comprehensive discussion as well as some in-depth prospects\\non the demand, design and deployment aspects of the wBAIM. We opine that wBAIM\\nwill be a recipe of the 6G wireless networks to build high-efficient,\\nsustainable, versatile, and extensible wireless intelligence for numerous\\npromising visions. Then, we present the core characteristics and principles to\\nguide the design of wBAIMs, and discuss the key aspects of developing wBAIMs\\nthrough identifying the differences between the existing BAIMs and the emerging\\nwBAIMs. Finally, related research directions and potential solutions are\\noutlined.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n')]\n",
+      "========================\n",
+      "Here are some of the latest developments in machine learning:\n",
+      "\n",
+      "1. [Foundation Model is Efficient Multimodal Multitask Model Selector](http://arxiv.org/pdf/2308.06262v1): This paper explores the problem of predicting the performance of pre-trained neural networks on different multi-modal tasks without fine-tuning them. The proposed method, called Efficient Multi-Task Model Selector (EMMS), uses large-scale foundation models to transform diverse label formats into a unified noisy label embedding. EMMS estimates a model's transferability through a simple weighted linear regression, achieving significant performance gains and speedup compared to existing methods.\n",
+      "\n",
+      "2. [Enhancing Network Management Using Code Generated by Large Language Models](http://arxiv.org/pdf/2308.06261v1): This paper introduces a novel approach to network management using large language models (LLMs) to generate task-specific code from natural language queries. The method addresses challenges related to explainability, scalability, and privacy by allowing network operators to inspect the generated code and focusing on application-specific requests combined with general program synthesis techniques.\n",
+      "\n",
+      "3. [Big AI Models for 6G Wireless Networks: Opportunities, Challenges, and Research Directions](http://arxiv.org/pdf/2308.06250v1): This paper discusses the potential of big artificial intelligence (AI) models in 6G wireless networks. The authors highlight the importance of designing and deploying wireless big AI models (wBAIMs) to enable intelligent communication, sensing, and computing. The paper provides insights into the demand, design principles, and research directions for wBAIMs in 6G wireless networks.\n",
+      "\n",
+      "These papers represent some of the recent advancements in machine learning and highlight the ongoing research in the field.\n"
+     ]
+    }
+   ],
+   "source": [
+    "agent = OpenAIAgent.from_tools(\n",
+    "    arxiv_tool.to_tool_list(),\n",
+    "    verbose=True,\n",
+    ")\n",
+    "print(agent.chat('what are the latest developments in machine learning'))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1c6a613f-c117-47d1-a02d-163ce64a2eba",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}