From c7e6e371f69657535976fb3c5fc31cbb2eada680 Mon Sep 17 00:00:00 2001 From: Jason Liu Date: Thu, 1 Feb 2024 10:48:03 -0500 Subject: [PATCH] Blog together (#102) --- docs/blog/.authors.yml | 2 +- docs/blog/posts/together.md | 85 +++++++++++++++++++++++++++++++++++++ mkdocs.yml | 4 +- 3 files changed, 88 insertions(+), 3 deletions(-) create mode 100644 docs/blog/posts/together.md diff --git a/docs/blog/.authors.yml b/docs/blog/.authors.yml index 0da18819..2236f8ea 100644 --- a/docs/blog/.authors.yml +++ b/docs/blog/.authors.yml @@ -2,4 +2,4 @@ authors: jxnl: name: Jason Liu description: Creator - avatar: https://pbs.twimg.com/profile_images/1724672723748638720/qOBwmkOI_400x400.jpg + avatar: https://avatars.githubusercontent.com/u/4852235 diff --git a/docs/blog/posts/together.md b/docs/blog/posts/together.md new file mode 100644 index 00000000..b71a526f --- /dev/null +++ b/docs/blog/posts/together.md @@ -0,0 +1,85 @@ +--- +draft: False +date: 2024-01-01 +slug: patching +tags: + - patching + - open source +authors: + - jxnl +--- + +# Structured Outputs with Together and Zod + +Open-source LLMS are gaining popularity, and the release of Togethers's Mistral model has made it possible to obtain structured outputs using JSON schema. Instead of relying on a model's default output mode, you can utilize JSON schema to obtain structured outputs. This approach is a time-saving alternative to extensive prompt engineering. + +By the end of this blog post, you will learn how to effectively utilize instructor with Togethers. But before we proceed, let's first explore the concept of patching. + +## Understanding Modes + +Instructor's patch enhances a openai api it with the following features, you can learn more about them [here](../../concepts/modes.md), for Togethers they support `JSON_SCHEMA` and `FUNCTIONS` modes. and with instructor we'll be able to use the following features: + +- `response_model` in `create` calls that returns a Zod schema +- `max_retries` in `create` calls that retries the call if it fails by using a backoff strategy + +## Anyscale + +The good news is that Anyscale employs the same OpenAI client, and its models support some of these output modes too! + +!!! note "Getting access" + + If you want to try this out for yourself check out the [Together Compute](https://together.ai/) website. You can get started [here](https://docs.together.ai/docs/quickstart). + +Let's explore one of the models available in Together's extensive collection! + +```ts +import Instructor from "@/instructor" +import OpenAI from "openai" +import { z } from "zod" + +const property = z.object({ + name: z.string(), + value: z.string() +}).describe("A property defined by a name and value") + +const UserSchema = z.object({ + age: z.number(), + name: z.string(), + properties: z.array(property) +}) + +const oai = new OpenAI({ + baseUrl='https://api.together.xyz', + apiKey: process.env.TOGETHER_API_KEY ?? undefined, +}) + +const client = Instructor({ + client: oai, + mode: "JSON_SCHEMA" +}) + +const user = await client.chat.completions.create({ + messages: [{ role: "user", content: "Harry Potter" }], + model: "mistralai/Mixtral-8x7B-Instruct-v0.1", + response_model: { schema: UserSchema }, + max_retries: 3 +}) + +console.log(user) +/** + * { + age: 17, + name: "Harry Potter", + properties: [ + { + name: "House", + value: "Gryffindor", + }, { + name: "Wand", + value: "Holly and Phoenix feather", + } + ], +} + */ +``` +You can find more information about Togethers's output mode support [here](https://docs.together.ai/docs/json-mode/). \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index d2b71adb..b2b5d8e3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -3,9 +3,9 @@ site_author: Jason Liu site_description: Enhancing OpenAI function calling with Zod repo_name: instructor repo_url: https://github.com/instructor-ai/instructor-js -site_url: https://jxnl.github.io/instructor-js/ +site_url: https://instructor-ai.github.io/instructor-js/ edit_uri: edit/main/docs/ -copyright: Copyright © 2024 Jason Liu +copyright: Copyright © 2024 Jason Liu theme: name: material icon: