Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Snowflake integration #1064

Merged
merged 14 commits into from
Sep 16, 2024
Merged

feat: Add Snowflake integration #1064

merged 14 commits into from
Sep 16, 2024

Conversation

medsriha
Copy link
Member

@medsriha medsriha commented Sep 8, 2024

Related Issues

  • fixes #issue-number

Proposed Changes:

Component that connects to a Snowflake database and retrieves data using a SQL query. This integration could be used standalone like such:

snowflake = SnowflakeRetriever(
    user="<ACCOUNT-USER>",
    account="<ACCOUNT-IDENTIFIER>",
    api_key=Secret.from_env_var("SNOWFLAKE_API_KEY"),
    warehouse="<WAREHOUSE-NAME>",
)

snowflake.run(query="""select * from table limit 10;"""")

Or within a pipeline for a text2SQL task, for example. See the attached example for reference.

Missing items:

How did you test it?

unit tests

Checklist

@github-actions github-actions bot added the type:documentation Improvements or additions to documentation label Sep 8, 2024
@medsriha medsriha marked this pull request as ready for review September 9, 2024 14:43
@medsriha medsriha requested a review from a team as a code owner September 9, 2024 14:43
@medsriha medsriha requested review from davidsbatista and removed request for a team September 9, 2024 14:43
Copy link
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work @medsriha!

I left a few comments, nothing big.

Regarding the tests please add serialization/deserialization of a pipeline with this component on it, see here an example: https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/google_vertex/tests/test_gemini.py#L261

@julian-risch julian-risch removed their request for review September 10, 2024 18:59
@medsriha
Copy link
Member Author

Thank you for reviewing this, @davidsbatista. I have also updated the object name from SnowflateRetriever to SnowflakeTableRetriever to clarify what this integration aims to achieve. Snowflake also offers a vector database, which we may consider integrating in the future.

Copy link
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@medsriha medsriha merged commit b47583f into main Sep 16, 2024
5 checks passed
@medsriha medsriha deleted the early-snow branch September 16, 2024 12:58
Amnah199 pushed a commit that referenced this pull request Oct 2, 2024
* initial commit

* add unit tests

* add pyproject.toml

* add pydoc config

* add CHANGELOG file

* update pyproject.toml

* lint file

* add example and fix lint

* update comments

* add header and trailing line

* update based on review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants