open_source_chain.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Huggingface models\n",
    "Langchain is not only about OpenAI´s GPT Models. You can also use free and open source models, for example from huggingface."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install torch==2.0.1\n",
    "!pip3 install transformers\n",
    "!pip3 install accelerate\n",
    "!pip3 install einops\n",
    "!pip3 install huggingface_hub\n",
    "!pip3 install langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import HuggingFaceHub\n",
    "from langchain import PromptTemplate, LLMChain"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You need a Huggingface account and API Key from Huggingface and create a token at: https://huggingface.co/settings/tokens.\n",
    "You can also set the token in the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = \"your-token\"\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()\n",
    "\n",
    "llm = HuggingFaceHub(repo_id=\"tiiuae/falcon-7b-instruct\", model_kwargs={\"max_length\":64, \"max_new_tokens\":100})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"\n",
    "You are a helpful bot that makes funny jokes about {topic}\n",
    "\n",
    "Answer: Let's think step by step.\n",
    "\"\"\"\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(template=template)\n",
    "chain = LLMChain(llm=llm, prompt=prompt_template)\n",
    "chain.run(\"Ducks\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Personal Opinion:\n",
    "The output is pretty BAD. The LLM is better when getting some context, which is written in a human like language."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"Answer the question as truthfully as possible using the provided text, and if the answer is not contained within the text below, say \"I don't know\"\n",
    "\n",
    "Context:\n",
    "{context}\n",
    "\n",
    "{query}\"\"\".strip()\n",
    "\n",
    "context = \"\"\"\n",
    "The men's high jump event at the 2020 Summer Olympics took place between 30 July and 1 August 2021 at the Olympic Stadium.\n",
    "33 athletes from 24 nations competed; the total possible number depended on how many nations would use universality places \n",
    "to enter athletes in addition to the 32 qualifying through mark or ranking (no universality places were used in 2021).\n",
    "Italian athlete Gianmarco Tamberi along with Qatari athlete Mutaz Essa Barshim emerged as joint winners of the event following\n",
    "a tie between both of them as they cleared 2.37m. Both Tamberi and Barshim agreed to share the gold medal in a rare instance\n",
    "where the athletes of different nations had agreed to share the same medal in the history of Olympics. \n",
    "Barshim in particular was heard to ask a competition official \"Can we have two golds?\" in response to being offered a \n",
    "'jump off'. Maksim Nedasekau of Belarus took bronze. The medals were the first ever in the men's high jump for Italy and \n",
    "Belarus, the first gold in the men's high jump for Italy and Qatar, and the third consecutive medal in the men's high jump\n",
    "for Qatar (all by Barshim). Barshim became only the second man to earn three medals in high jump, joining Patrik Sjöberg\n",
    "of Sweden (1984 to 1992).\"\"\"\n",
    "\n",
    "\n",
    "prompt  = PromptTemplate(\n",
    "    input_variables=[\"context\", \"query\"],\n",
    "    template=template\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"Who won the 2020 Summer Olympics men's high jump?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = LLMChain(llm=llm, prompt=prompt)\n",
    "chain.run({\"query\": query, \"context\": context})"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}