Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check writer pipeline #1201

Closed
wants to merge 32 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
964c852
feat: initial version of Writer provider
yanomaly Nov 11, 2024
320dfb9
test: initialize tests for Writer provider
yanomaly Nov 11, 2024
98effd1
feat: addition implementations for Writer provider
yanomaly Nov 11, 2024
c160d02
feat: implement basic formatting for Writer
yanomaly Nov 12, 2024
f7d0e5f
test: add tests for basic formatting
yanomaly Nov 12, 2024
c352fb8
test: add tests for Writer provider base use-cases
yanomaly Nov 12, 2024
5bd2b9f
feat: add writer multiple tools handling
yanomaly Nov 15, 2024
e5af98a
feat: add writer reasks handler
yanomaly Nov 15, 2024
d888a27
test: add tests for async retries
yanomaly Nov 15, 2024
54a4744
test: add tests for patch
yanomaly Nov 15, 2024
7ab10f9
test: add validators tests
yanomaly Nov 15, 2024
d0ab970
feat: add streaming sync and async support (with bugs)
yanomaly Nov 15, 2024
9c555a5
test: add tests for streaming
yanomaly Nov 15, 2024
23c1e23
Merge branch 'main' into feat/writer-integration
ivanleomk Nov 17, 2024
6d0a686
fix: adding pyproject.toml extra for writer-sdk
ivanleomk Nov 17, 2024
eb41907
fix: remove writer function so that we just use .content instead
ivanleomk Nov 17, 2024
d367810
fix: remove unecessary tests
ivanleomk Nov 17, 2024
320e652
fix: fixed up partials so that it accounts for writer returning the e…
ivanleomk Nov 17, 2024
a5084c9
feat: add writer article
ivanleomk Nov 17, 2024
f28b4a4
fix: changed copy
ivanleomk Nov 17, 2024
589e7b2
fix: update poetry.lock
ivanleomk Nov 17, 2024
bb0143c
fix: remove unused import
ivanleomk Nov 17, 2024
b95032f
fix: update types
ivanleomk Nov 17, 2024
4d87b92
fix: add writer dep for pyright
ivanleomk Nov 17, 2024
d549420
test: add eval tests
yanomaly Nov 18, 2024
5d436ca
test: edit streaming tests
yanomaly Nov 18, 2024
b84719c
test: add multitask tests
yanomaly Nov 18, 2024
fc1e458
docs: enance Writer article
yanomaly Nov 19, 2024
9f1ab9f
Merge branch 'main' into feat/writer-integration
ivanleomk Nov 20, 2024
a0b8dc5
fix : adding writer support article, fixed the announcement article a…
ivanleomk Nov 21, 2024
8652b9c
fix: migrate types
ivanleomk Nov 21, 2024
c2924c4
fix: remove missing import
ivanleomk Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/blog/.authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,8 @@ authors:
name: Thierry Jean
description: Contributor
avatar: https://avatars.githubusercontent.com/u/68975210?v=4
url: https://www.linkedin.com/in/thierry-jean/
url: https://www.linkedin.com/in/thierry-jean/
yanomaly:
name: Yan
description: Contributor
avatar: https://avatars.githubusercontent.com/u/87994542?v=4
162 changes: 162 additions & 0 deletions docs/blog/posts/writer-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
authors:
- ivanleomk
- yanomaly
categories:
- Writer SDK
comments: true
date: 2024-11-19
description: Announcing Writer integration with Instructor for structured outputs and enterprise AI workflows
draft: false
slug: writer-support
tags:
- Writer
- Enterprise AI
- Integrations
---

# Structured Outputs with Writer now supported

We're excited to announce that `instructor` now supports [Writer](https://writer.com)'s enterprise-grade LLMs, including their latest Palmyra X 004 model. This integration enables structured outputs and enterprise AI workflows with Writer's powerful language models.

## Getting Started

First, make sure that you've signed up for an account on [Writer](https://writer.com) and obtained an API key. Once you've done so, install `instructor` with Writer support by running `pip install instructor[writer]` in your terminal.

Make sure to set the `WRITER_API_KEY` environment variable with your Writer API key or pass it as an argument to the `Writer` constructor.

<!-- more -->

```python
import instructor
from writerai import Writer
from pydantic import BaseModel

# Initialize Writer client
client = instructor.from_writer(Writer(api_key="your API key"))


class User(BaseModel):
name: str
age: int


# Extract structured data
user = client.chat.completions.create(
model="palmyra-x-004",
messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
response_model=User,
)

print(user)
#> name='John' age=30
```

!!! note

If you'd like to use the Async version of the Writer client, you can do so by using `instructor.from_writer(AsyncWriter())`.

We also support streaming with the Writer client using our `create_partial` method. This allows you to process responses incrementally as they arrive.

This is particularly valuable for maintaining responsive applications and delivering a smooth user experience, especially when dealing with larger responses so that users can see immediate results.

```python
import instructor
from writerai import Writer
from pydantic import BaseModel

# Initialize Writer client
client = instructor.from_writer(Writer())


text_block = """
In our recent online meeting, participants from various backgrounds joined to discuss the upcoming tech conference. The names and contact details of the participants were as follows:

- Name: John Doe, Email: [email protected], Twitter: @TechGuru44
- Name: Jane Smith, Email: [email protected], Twitter: @DigitalDiva88
- Name: Alex Johnson, Email: [email protected], Twitter: @CodeMaster2023

During the meeting, we agreed on several key points. The conference will be held on March 15th, 2024, at the Grand Tech Arena located at 4521 Innovation Drive. Dr. Emily Johnson, a renowned AI researcher, will be our keynote speaker.

The budget for the event is set at $50,000, covering venue costs, speaker fees, and promotional activities. Each participant is expected to contribute an article to the conference blog by February 20th.

A follow-up meetingis scheduled for January 25th at 3 PM GMT to finalize the agenda and confirm the list of speakers.
"""


class User(BaseModel):
name: str
email: str
twitter: str


class MeetingInfo(BaseModel):
date: str
location: str
budget: int
deadline: str


PartialMeetingInfo = instructor.Partial[MeetingInfo]


extraction_stream = client.chat.completions.create(
model="palmyra-x-004",
messages=[
{
"role": "user",
"content": f"Get the information about the meeting and the users {text_block}",
},
],
response_model=PartialMeetingInfo,
stream=True,
) # type: ignore


for obj in extraction_stream:
print(obj)
#> date='March 15th, 2024' location='' budget=None deadline=None
#> date='March 15th, 2024' location='Grand Tech Arena, 4521 Innovation' budget=None deadline=None
#> date='March 15th, 2024' location='Grand Tech Arena, 4521 Innovation Drive' budget=50000 eadline='February 20th'
```

As with all our integrations, `instructor` ships with the ability to automatically retry requests that happen due to schema validation without you having to do anything.

```python
import instructor
from typing import Annotated
from writerai import Writer
from pydantic import BaseModel, AfterValidator, Field

#Initialize Writer client
client = instructor.from_writer(Writer())


#Example of model, that may require usage of retries
def uppercase_validator(v):
if v.islower():
raise ValueError("Name must be in uppercase")
return v


class User(BaseModel):
name: Annotated[str, AfterValidator(uppercase_validator)] = Field(
..., description="The name of the user"
)
age: int


user = client.chat.completions.create(
model="palmyra-x-004",
messages=[{"role": "user", "content": "Extract: jason is 12"}],
response_model=User,
max_retries=3,
)

print(user)
#> name='JASON' age=12
```

This was a sneak peek into the things that you can do with Writer and `instructor` - from classification of text to sentimen analysis and more.

We're excited to see what you build with `instructor` and Writer. If you have any other questions about writer, do check out the [Writer Documentation](https://dev.writer.com/introduction) for the API sdk.
5 changes: 3 additions & 2 deletions docs/integrations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,12 @@ Instructor supports a wide range of AI model providers, each with their own capa
- [Anthropic](./anthropic.md) - Claude and Claude 2 models
- [Google](./google.md) - PaLM and Gemini models
- [Vertex AI](./vertex.md) - Google Cloud's AI platform
- [Cohere](./cohere.md) - Command and other Cohere models
- [Cohere](./cohere.md) - Command-R and other Cohere models
- [Groq](./groq.md) - High-performance inference platform
- [Mistral](./mistral.md) - Mistral's hosted models
- [Fireworks](./fireworks.md) - High-performance model inference
- [Cerebras](./cerebras.md) - AI accelerator platform
- [Cerebras](./cerebras.md) - Llama-3-70B and other Open Source Models at blazing fast inference speeds
- [Writer](./writer.md) - Palmyra-X-004 and other Writer models

### Model Management

Expand Down
181 changes: 181 additions & 0 deletions docs/integrations/writer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
---
title: Structured Outputs with Writer
description: Learn how to use Writer for structured outputs using their latest Palmyra-X-004 model for more reliable system outputs
---

# Structured Outputs with Writer

This guide demonstrates how to use Writer for structured outputs using their latest Palmyra-X-004 model for more reliable system outputs.

You'll need to sign up for an account and get an API key. You can do that [here](https://writer.com).

```bash
export WRITER_API_KEY=<your-api-key-here>
pip install "instructor[writer]"
```

## Palmyra-X-004

Writer supports structured outputs with their latest Palmyra-X-004 model that introduces tool calling functionality

### Sync Example

```python
import instructor
from writerai import Writer
from pydantic import BaseModel

# Initialize Writer client
client = instructor.from_writer(Writer(api_key="your API key"))


class User(BaseModel):
name: str
age: int


# Extract structured data
user = client.chat.completions.create(
model="palmyra-x-004",
messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
response_model=User,
)

print(user)
#> name='John' age=30
```

### Async Example

```python
import instructor
from writerai import AsyncWriter
from pydantic import BaseModel

# Initialize Writer client
client = instructor.from_writer(AsyncWriter())


class User(BaseModel):
name: str
age: int


async def extract_user():
# Extract structured data
user = await client.chat.completions.create(
model="palmyra-x-004",
messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
response_model=User,
)

print(user)
# > name='John' age=30


if __name__ == "__main__":
import asyncio

asyncio.run(extract_user())
```

## Nested Objects

Writer also supports nested objects, which is useful for extracting data from more complex responses.

```python
import instructor
from writerai import Writer
from pydantic import BaseModel

# Initialize Writer client
client = instructor.from_writer(Writer())


class Address(BaseModel):
street: str
city: str
country: str


class User(BaseModel):
name: str
age: int
addresses: list[Address]


# Create structured output with nested objects
user = client.chat.completions.create(
model="palmyra-x-004",
messages=[
{
"role": "user",
"content": """
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
},
],
response_model=User,
)
print(user)
#> {
#> 'name': 'Jason',
#> 'age': 25,
#> 'addresses': [
#> {
#> 'street': '123 Main St',
#> 'city': 'New York',
#> 'country': 'USA'
#> },
#> {
#> 'street': '456 Beach Rd',
#> 'city': 'Miami',
#> 'country': 'USA'
#> }
#> ]
#> }
```

## Streaming Support

Instructor has two main ways that you can use to stream responses out

1. **Iterables**: These are useful when you'd like to stream a list of objects of the same type (Eg. use structured outputs to extract multiple users)
2. **Partial Streaming**: This is useful when you'd like to stream a single object and you'd like to immediately start processing the response as it comes in.

We currently support streaming for Writer with native tool for both methods listed above.

### Partial Streaming

```python
import instructor
from writerai import Writer
from pydantic import BaseModel

client = instructor.from_writer(Writer())


class Person(BaseModel):
name: str
age: int


resp = client.chat.completions.create_partial(
model="palmyra-x-004",
messages=[
{
"role": "user",
"content": "Ivan is 27 and lives in Singapore",
}
],
response_model=Person,
)

for person in resp:
print(person)
# > name=None age=None
# > name='Ivan' age=None
# > name='Ivan' age=27
```
5 changes: 5 additions & 0 deletions instructor/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,8 @@
from .client_vertexai import from_vertexai

__all__ += ["from_vertexai"]

if importlib.util.find_spec("writerai") is not None:
from .client_writer import from_writer

__all__ += ["from_writer"]
Loading
Loading