Skip to content

Commit

Permalink
fix: updated mistral and vertexai
Browse files Browse the repository at this point in the history
  • Loading branch information
@ivanleomk
ivanleomk committed Nov 19, 2024
1 parent 7bb4a24 commit adccac6
Show file tree
Hide file tree
Showing 3 changed files with 63 additions and 202 deletions.
2 changes: 1 addition & 1 deletion docs/integrations/google.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "Complete guide to using Instructor with Google's Gemini models. Le

# Structured outputs with Google/Gemini, a complete guide w/ instructor

Instructor supports the VertexAI and the Google.GenerativeAI libraries. This guide will show you how to use Instructor with the Google.GenerativeAI library.
This guide will show you how to use Instructor with the Google.GenerativeAI library. We recommend this library for most users as it's significantly easier to get started with.

## Google.GenerativeAI

Expand Down
62 changes: 21 additions & 41 deletions docs/integrations/mistral.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,54 +20,34 @@ Mistral Large is the flagship model from Mistral AI, supporting 32k context wind

By the end of this blog post, you will learn how to effectively utilize Instructor with Mistral Large.

<!-- more -->

## Patching

Instructor's patch enhances the mistral api with the following features:

- `response_model` in `create` calls that returns a pydantic model
- `max_retries` in `create` calls that retries the call if it fails by using a backoff strategy

!!! note "Learn More"

To learn more, please refer to the [docs](../index.md). To understand the benefits of using Pydantic with Instructor, visit the tips and tricks section of the [why use Pydantic](../why.md) page.

## Mistral Client

The Mistral client employs a different client than OpenAI, making the patching process slightly different than other examples

!!! note "Getting access"
```python
import os
from pydantic import BaseModel
from mistralai import Mistral
from instructor import from_mistral, Mode

If you want to try this out for yourself check out the [Mistral AI](https://mistral.ai/) website. You can get started [here](https://docs.mistral.ai/).

```python
import instructor
class UserDetails(BaseModel):
name: str
age: int

from pydantic import BaseModel
from mistralai.client import MistralClient

# enables `response_model` in chat call
client = MistralClient()
client = Mistral(api_key=os.environ.get("MISTRAL_API_KEY"))

patched_chat = instructor.from_openai(create=client.chat, mode=instructor.Mode.MISTRAL_TOOLS)
instructor_client = from_mistral(
client=client,
model="mistral-large-latest",
mode=Mode.MISTRAL_TOOLS,
max_tokens=1000,
)

if __name__ == "__main__":
resp = instructor_client.messages.create(
response_model=UserDetails,
messages=[{"role": "user", "content": "Jason is 10"}],
temperature=0,
)

class UserDetails(BaseModel):
name: str
age: int
print(resp)

resp = patched_chat(
model="mistral-large-latest",
response_model=UserDetails,
messages=[
{
"role": "user",
"content": f'Extract the following entities: "Jason is 20"',
},
],
)
print(resp)
#> name='Jason' age=20
```
201 changes: 41 additions & 160 deletions docs/integrations/vertex.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,209 +9,90 @@ Google Cloud's Vertex AI provides enterprise-grade AI capabilities with robust s

## Quick Start

Install Instructor with Vertex AI support:
Install Instructor with Vertex AI support. You can do so by running the command below.

```bash
pip install "instructor[vertex]"
```

You'll also need the Google Cloud SDK and proper authentication:

```bash
pip install google-cloud-aiplatform
pip install "instructor[vertexai]"
```

## Simple User Example (Sync)

```python
from vertexai.language_models import TextGenerationModel
import instructor
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
from pydantic import BaseModel

# Initialize the model
model = TextGenerationModel.from_pretrained("text-bison@001")
vertexai.init()

# Enable instructor patches
client = instructor.from_vertex(model)

class User(BaseModel):
name: str
age: int

# Create structured output
user = client.predict(
prompt="Extract: Jason is 25 years old",

client = instructor.from_vertexai(
client=GenerativeModel("gemini-1.5-pro-preview-0409"),
mode=instructor.Mode.VERTEXAI_TOOLS,
)

# note that client.chat.completions.create will also work
resp = client.create(
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)

print(user) # User(name='Jason', age=25)
print(resp)
#> User(name='Jason', age=25)
```

## Simple User Example (Async)

```python
from vertexai.language_models import TextGenerationModel
import instructor
import vertexai # type: ignore
from vertexai.generative_models import GenerativeModel # type: ignore
from pydantic import BaseModel
import asyncio

# Initialize the model
model = TextGenerationModel.from_pretrained("text-bison@001")
vertexai.init()

# Enable instructor patches
client = instructor.from_vertex(model)

class User(BaseModel):
name: str
age: int


client = instructor.from_vertexai(
client=GenerativeModel("gemini-1.5-pro-preview-0409"),
mode=instructor.Mode.VERTEXAI_TOOLS,
_async=True,
)

async def extract_user():

async def extract_user():
user = await client.predict_async(
prompt="Extract: Jason is 25 years old",
user = await client.create(
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
return user


# Run async function
user = asyncio.run(extract_user())
print(user) # User(name='Jason', age=25)
```

## Nested Example

```python
from pydantic import BaseModel
from typing import List

class Address(BaseModel):
street: str
city: str
country: str

class User(BaseModel):
name: str
age: int
addresses: List[Address]

# Create structured output with nested objects
user = client.predict(
prompt="""
Extract: Jason is 25 years old.
He lives at 123 Main St, New York, USA
and has a summer house at 456 Beach Rd, Miami, USA
""",
response_model=User,
)

print(user) # User with nested Address objects
```

## Partial Streaming Example

Note: Vertex AI's current API does not support partial streaming of responses. The streaming functionality returns complete responses in chunks rather than partial objects. We recommend using the standard synchronous or asynchronous methods for structured output generation.

## Iterable Example

```python
from typing import List

class User(BaseModel):
name: str
age: int

# Extract multiple users from text
users = client.predict_iterable(
prompt="""
Extract users:
1. Jason is 25 years old
2. Sarah is 30 years old
3. Mike is 28 years old
""",
response_model=User,
)

for user in users:
print(user) # Prints each user as it's extracted
```

## Instructor Hooks

Instructor provides several hooks to customize behavior:

### Validation Hook

```python
from instructor import Instructor

def validation_hook(value, retry_count, exception):
print(f"Validation failed {retry_count} times: {exception}")
return retry_count < 3 # Retry up to 3 times

instructor.patch(client, validation_hook=validation_hook)
```

### Mode Hooks

```python
from instructor import Mode

# Use different modes for different scenarios
client = instructor.patch(client, mode=Mode.JSON) # JSON mode
client = instructor.patch(client, mode=Mode.TOOLS) # Tools mode
client = instructor.patch(client, mode=Mode.MD_JSON) # Markdown JSON mode
```

### Custom Retrying

```python
from instructor import RetryConfig

client = instructor.patch(
client,
retry_config=RetryConfig(
max_retries=3,
on_retry=lambda *args: print("Retrying..."),
)
)
```

## Available Models

Vertex AI offers several model options:
- PaLM 2 for Text (text-bison)
- PaLM 2 for Chat (chat-bison)
- Codey for Code Generation
- Enterprise-specific models
- Custom-trained models

## Best Practices

1. **Model Selection**
- Choose model based on enterprise requirements
- Consider security and compliance needs
- Monitor quota and costs
- Use appropriate model versions

2. **Optimization Tips**
- Structure prompts effectively
- Use appropriate temperature settings
- Implement caching strategies
- Monitor API usage

3. **Error Handling**
- Implement proper validation
- Handle quota limits gracefully
- Monitor model responses
- Use appropriate timeout settings

## Common Use Cases

- Enterprise Data Processing
- Secure Content Generation
- Document Analysis
- Compliance-Aware Processing
- Large-Scale Deployments

## Related Resources

- [Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs)
Expand Down

0 comments on commit adccac6

Please sign in to comment.