L5-text-generation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e4f357d1-bbc9-4848-828c-00b36144ebdc",
   "metadata": {},
   "source": [
    "## Lesson 5: Text Generation with Vertex AI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a416b662-c645-4048-8557-fd0f66d9818c",
   "metadata": {},
   "source": [
    "#### Project environment setup\n",
    "\n",
    "- Load credentials and relevant Python Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a9d7e7fc-0c16-41c5-ba3d-988fb011deb5",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "from utils import authenticate\n",
    "credentials, PROJECT_ID = authenticate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ff77bf9c-93e6-4281-9775-f7e87935b37a",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "REGION = 'us-central1'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02fb2abd-f5d3-4fa0-8c4c-8d516df20c7f",
   "metadata": {},
   "source": [
    "### Prompt the model\n",
    "- We'll import a language model that has been trained to handle a variety of natural language tasks, `text-bison@001`.\n",
    "- For multi-turn dialogue with a language model, you can use, `chat-bison@001`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4bc8793c-bae4-405e-9dcc-2d5657c7486b",
   "metadata": {
    "height": 81
   },
   "outputs": [],
   "source": [
    "import vertexai\n",
    "vertexai.init(project=PROJECT_ID, \n",
    "              location=REGION, \n",
    "              credentials = credentials)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f253273a-a0bd-41c8-bf64-138666578504",
   "metadata": {
    "height": 40
   },
   "outputs": [],
   "source": [
    "from vertexai.language_models import TextGenerationModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "da149416-7fea-4bb3-b872-f64200f9b19c",
   "metadata": {
    "height": 57
   },
   "outputs": [],
   "source": [
    "generation_model = TextGenerationModel.from_pretrained(\n",
    "    \"text-bison@001\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36ac4f25-7818-49fc-b9e9-097886da8e82",
   "metadata": {},
   "source": [
    "#### Question Answering\n",
    "- You can ask an open-ended question to the language model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c3a1b5ab-1bb8-4882-93e8-cc9381b18aad",
   "metadata": {
    "height": 57
   },
   "outputs": [],
   "source": [
    "prompt = \"I'm a high school student. \\\n",
    "Recommend me a programming activity to improve my skills.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "84aca913-e2c1-4ad5-b8ee-4945c9546102",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* **Write a program to solve a problem you're interested in.** This could be anything from a game to a tool to help you with your studies. The important thing is that you're interested in the problem and that you're motivated to solve it.\n",
      "* **Take a programming course.** There are many online and offline courses available, so you can find one that fits your schedule and learning style.\n",
      "* **Join a programming community.** There are many online and offline communities where you can connect with other programmers and learn from each other.\n",
      "* **Read programming books and articles.** There is a\n"
     ]
    }
   ],
   "source": [
    "print(generation_model.predict(prompt=prompt).text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd540471-cda7-43b2-aee3-b84cb1234c1a",
   "metadata": {},
   "source": [
    "#### Classify and elaborate\n",
    "- For more predictability of the language model's response, you can also ask the language model to choose among a list of answers and then elaborate on its answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e1491242-77fb-4d8e-8bda-04f4894fdd83",
   "metadata": {
    "height": 115
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"I'm a high school student. \\\n",
    "Which of these activities do you suggest and why:\n",
    "a) learn Python\n",
    "b) learn Javascript\n",
    "c) learn Fortran\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "28880988-e191-4991-baa8-fb7c89cc4874",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I would suggest learning Python. Python is a general-purpose programming language that is easy to learn and has a wide range of applications. It is used in a variety of fields, including web development, data science, and machine learning. Python is also a popular language for beginners, as it has a large community of support and resources available.\n"
     ]
    }
   ],
   "source": [
    "print(generation_model.predict(prompt=prompt).text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913e102f-e4d7-4ac8-9b76-3a0c1b3790b9",
   "metadata": {},
   "source": [
    "#### Extract information and format it as a table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "efdfe0f7-678c-4ad6-bdcb-4cc299512570",
   "metadata": {
    "height": 346
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\" A bright and promising wildlife biologist \\\n",
    "named Jesse Plank (Amara Patel) is determined to make her \\\n",
    "mark on the world. \n",
    "Jesse moves to Texas for what she believes is her dream job, \n",
    "only to discover a dark secret that will make \\\n",
    "her question everything. \n",
    "In the new lab she quickly befriends the outgoing \\\n",
    "lab tech named Maya Jones (Chloe Nguyen), \n",
    "and the lab director Sam Porter (Fredrik Johansson). \n",
    "Together the trio work long hours on their research \\\n",
    "in a hope to change the world for good. \n",
    "Along the way they meet the comical \\\n",
    "Brenna Ode (Eleanor Garcia) who is a marketing lead \\\n",
    "at the research institute, \n",
    "and marine biologist Siri Teller (Freya Johansson).\n",
    "\n",
    "Extract the characters, their jobs \\\n",
    "and the actors who played them from the above message as a table\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "92efacae-afe4-4cd2-9d6d-cb565d102ccc",
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "| Character | Job | Actor |\n",
      "|---|---|---|\n",
      "| Jesse Plank | Wildlife Biologist | Amara Patel |\n",
      "| Maya Jones | Lab Tech | Chloe Nguyen |\n",
      "| Sam Porter | Lab Director | Fredrik Johansson |\n",
      "| Brenna Ode | Marketing Lead | Eleanor Garcia |\n",
      "| Siri Teller | Marine Biologist | Freya Johansson |\n"
     ]
    }
   ],
   "source": [
    "response = generation_model.predict(prompt=prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d80be8ca-4732-4b39-a5e6-5db1d3b02218",
   "metadata": {},
   "source": [
    "- You can copy-paste the text into a markdown cell to see if it displays a table."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18816c8e-a473-4eb5-8983-7c5f80f8aa15",
   "metadata": {},
   "source": [
    "| Character | Job | Actor |\n",
    "|---|---|---|\n",
    "| Jesse Plank | Wildlife Biologist | Amara Patel |\n",
    "| Maya Jones | Lab Tech | Chloe Nguyen |\n",
    "| Sam Porter | Lab Director | Fredrik Johansson |\n",
    "| Brenna Ode | Marketing Lead | Eleanor Garcia |\n",
    "| Siri Teller | Marine Biologist | Freya Johansson |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87b9c406-2b3f-4fb0-a181-201091013864",
   "metadata": {},
   "source": [
    "### Adjusting Creativity/Randomness\n",
    "- You can control the behavior of the language model's decoding strategy by adjusting the temperature, top-k, and top-n parameters.\n",
    "- For tasks for which you want the model to consistently output the same result for the same input, (such as classification or information extraction), set temperature to zero.\n",
    "- For tasks where you desire more creativity, such as brainstorming, summarization, choose a higher temperature (up to 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f629e383-b4ad-4fa6-a949-435dd1f04da6",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "temperature = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "84519df4-9b6f-471c-8183-27cc4a151d7d",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "prompt = \"Complete the sentence: \\\n",
    "As I prepared the picture frame, \\\n",
    "I reached into my toolkit to fetch my:\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2a2fe53e-d885-411c-bb2e-c3940884d30f",
   "metadata": {
    "height": 81
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt,\n",
    "    temperature=temperature,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2be86a8a-a58f-4b9e-bd66-9eced64e549c",
   "metadata": {
    "height": 47
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[temperature = 0.0]\n",
      "As I prepared the picture frame, I reached into my toolkit to fetch my hammer.\n"
     ]
    }
   ],
   "source": [
    "print(f\"[temperature = {temperature}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ac5bd8e5-8fdc-4d94-acf9-10b7d3b34ba8",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "temperature = 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "f0bf346d-2617-460e-bc47-f331b92ed053",
   "metadata": {
    "height": 81
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt,\n",
    "    temperature=temperature,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "67e25ae3-a1ae-4379-ad32-8d79e52b7590",
   "metadata": {
    "height": 47
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[temperature = 1.0]\n",
      "As I prepared the picture frame, I reached into my toolkit to fetch my hammer. The hammer is a tool that is used to bang in nails, and it is a common tool to use when preparing a picture frame.\n"
     ]
    }
   ],
   "source": [
    "print(f\"[temperature = {temperature}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fea806c4-f955-4434-89a5-57fd9083e9d3",
   "metadata": {},
   "source": [
    "#### Top P\n",
    "- Top p: sample the minimum set of tokens whose probabilities add up to probability `p` or greater.\n",
    "- The default value for `top_p` is `0.95`.\n",
    "- If you want to adjust `top_p` and `top_k` and see different results, remember to set `temperature` to be greater than zero, otherwise the model will always choose the token with the highest probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "af98a491-6da8-4cca-bf16-85f9547e7fe5",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "top_p = 0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2e195211-60ea-4103-b083-19e78a26c920",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "prompt = \"Write an advertisement for jackets \\\n",
    "that involves blue elephants and avocados.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "28093674-1a83-4abb-999a-030df75b2125",
   "metadata": {
    "height": 98
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt, \n",
    "    temperature=0.9, \n",
    "    top_p=top_p,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "30a4ce9d-7520-44bc-af24-dc1b6ad61b50",
   "metadata": {
    "height": 47
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[top_p = 0.2]\n",
      "**Introducing the new Blue Elephant Avocado Jacket!**\n",
      "\n",
      "This jacket is the perfect way to show your love of both blue elephants and avocados. It's made of soft, durable fabric and features a cozy lining. The front has a large blue elephant print, and the back has a smaller avocado print.\n",
      "\n",
      "The Blue Elephant Avocado Jacket is perfect for any occasion. Wear it to a concert, a party, or just a casual day out. You're sure to turn heads wherever you go!\n",
      "\n",
      "**Order your Blue Elephant Avocado Jacket today!**\n"
     ]
    }
   ],
   "source": [
    "print(f\"[top_p = {top_p}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02817243-ef3f-440a-8846-33dc46963b41",
   "metadata": {},
   "source": [
    "#### Top k\n",
    "- The default value for `top_k` is `40`.\n",
    "- You can set `top_k` to values between `1` and `40`.\n",
    "- The decoding strategy applies `top_k`, then `top_p`, then `temperature` (in that order)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "f6603255-1e19-47ce-8a77-634b33c1e99f",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "top_k = 20\n",
    "top_p = 0.7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ef545757-b37c-4b39-b7e9-3ebc4ec1d910",
   "metadata": {
    "height": 115
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt, \n",
    "    temperature=0.9, \n",
    "    top_k=top_k,\n",
    "    top_p=top_p,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "d09c4f8d-df9d-4d61-9ab6-19958f8b4129",
   "metadata": {
    "height": 47
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[top_p = 0.7]\n",
      "**Introducing the new Blue Elephant Avocado Jacket!**\n",
      "\n",
      "This jacket is the perfect way to show your love of both blue elephants and avocados. It's made of soft, durable fabric and features a fun, all-over print of blue elephants and avocados. The jacket is also fully lined for extra warmth.\n",
      "\n",
      "Whether you're wearing it to the park, to the grocery store, or just hanging out at home, the Blue Elephant Avocado Jacket is sure to turn heads. So what are you waiting for? Order yours today!\n",
      "\n",
      "**Here are some of the features of the Blue Elephant Avocado Jacket:**\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"[top_p = {top_p}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "005c5ff2-03f0-4dcd-a808-c92c9c5bbb0a",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}