Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Amazon Bedrock] Add AmazonBedrockEmbedder #181

Closed
4 tasks done
julian-risch opened this issue Jan 8, 2024 · 2 comments
Closed
4 tasks done

[Amazon Bedrock] Add AmazonBedrockEmbedder #181

julian-risch opened this issue Jan 8, 2024 · 2 comments
Assignees
Labels
feature request Ideas to improve an integration P1
Milestone

Comments

@julian-risch
Copy link
Member

julian-risch commented Jan 8, 2024

In addition to the recently merged AmazonBedrockGenerator #153, we need an AmazonBedrockEmbedder for the embedding models.

Tasks

Preview Give feedback
@lambda-science
Copy link
Contributor

Just a heads up for anyone really wanting to use AWS Bedrock Embedding with Haystack 2.0 but can't because it's not implemented yet. It's possible to be done in a hacky-way with LiteLLM Proxy https://litellm.vercel.app/docs/simple_proxy

If you have a docker-compose hosting your Haystack API you can do something like this. You can also do it with this sample configuration:
docker-compose.yml:

  litellm:
    image: ghcr.io/berriai/litellm:main-v1.17.2
    ports:
      - "8001:8001"
    environment:
      - AWS_PROFILE=${AWS_PROFILE_NAME}
      - AWS_REGION=${AWS_REGION_NAME}
      - LITELLM_LOG=INFO
    volumes:
      - ./litellm-config.yaml:/app/config.yaml
      - ~/.aws/:/root/.aws/
    command: [ "--config", "/app/config.yaml", "--port", "8001", "--num_workers", "8" ]

litellm-config.yaml

model_list:
  - model_name: bedrock-titan-embed-text-v1
    litellm_params:
      model: bedrock/amazon.titan-embed-text-v1

litellm_settings:
  drop_params: True

And in your Haystack 2.x pipeline:

from haystack.components.embedders import OpenAIDocumentEmbedder
from haystack.components.embedders import OpenAITextEmbedder
API_BASE_URL = "http://litellm:8001"
document_embedder = OpenAIDocumentEmbedder(api_base_url=API_BASE_URL, model_name="bedrock-titan-embed-text-v1")
query_embedder = OpenAITextEmbedder(api_base_url=API_BASE_URL, model_name="bedrock-titan-embed-text-v1")

Not optimal because it requeries an external container but it's a "quick win" as we say ahah
I'm not sure I have the programming skills to do a proper implementation in Haystack of Bedrock embedding unfortunately :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Ideas to improve an integration P1
Projects
None yet
Development

No branches or pull requests

6 participants