Skip to content

Commit

Permalink
Merge branch 'main' into add-supabase-vectordb
Browse files Browse the repository at this point in the history
  • Loading branch information
masci authored Feb 1, 2024
2 parents 6402ed7 + 0d15e36 commit a999960
Show file tree
Hide file tree
Showing 227 changed files with 8,796 additions and 1,113 deletions.
26 changes: 26 additions & 0 deletions .github/ISSUE_TEMPLATE/breaking-change-proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
name: Breaking change proposal
about: Track a breaking change for an existing integration
title: ''
labels: breaking change
assignees: ''

---

## Summary and motivation

Briefly explain how the change is breaking and why is needed.

## Checklist

```[tasklist]
### Tasks
- [ ] The changes are merged in the `main` branch (Code + Docstrings)
- [ ] New package version declares the breaking change
- [ ] The package has been released on PyPI
- [ ] Docs at https://docs.haystack.deepset.ai/ were updated
- [ ] Integration tile on https://github.com/deepset-ai/haystack-integrations was updated
- [ ] Notebooks on https://github.com/deepset-ai/haystack-cookbook were updated (if needed)
- [ ] Tutorials on https://github.com/deepset-ai/haystack-tutorials were updated (if needed)
- [ ] Articles on https://github.com/deepset-ai/haystack-home/tree/main/content were updated (if needed)
```
6 changes: 4 additions & 2 deletions .github/ISSUE_TEMPLATE/new-integration-proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@ Also, if there's any new terminology involved, define it here.
## Checklist

If the request is accepted, ensure the following checklist is complete before closing this issue.

- [ ] The code was merged in the `main` branch
```[tasklist]
### Tasks
- [ ] The code is documented with docstrings and was merged in the `main` branch
- [ ] Docs are published at https://docs.haystack.deepset.ai/
- [ ] There is a Github workflow running the tests for the integration nightly and at every PR
- [ ] A label named like `integration:<your integration name>` has been added to this repo
Expand All @@ -31,3 +32,4 @@ If the request is accepted, ensure the following checklist is complete before cl
- [ ] The integration has been listed in the [Inventory section](https://github.com/deepset-ai/haystack-core-integrations#inventory) of this repo README
- [ ] There is an example available to demonstrate the feature
- [ ] The feature was announced through social media
```
42 changes: 31 additions & 11 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ integration:amazon-bedrock:
- any-glob-to-any-file: "integrations/amazon_bedrock/**/*"
- any-glob-to-any-file: ".github/workflows/amazon_bedrock.yml"

integration:amazon-sagemaker:
- changed-files:
- any-glob-to-any-file: "integrations/amazon_sagemaker/**/*"
- any-glob-to-any-file: ".github/workflows/amazon_sagemaker.yml"

integration:astra:
- changed-files:
- any-glob-to-any-file: "integrations/astra/**/*"
- any-glob-to-any-file: ".github/workflows/astra.yml"

integration:chroma:
- changed-files:
- any-glob-to-any-file: "integrations/chroma/**/*"
Expand Down Expand Up @@ -44,35 +54,45 @@ integration:jina:
- any-glob-to-any-file: "integrations/jina/**/*"
- any-glob-to-any-file: ".github/workflows/jina.yml"

integration:llama_cpp:
- changed-files:
- any-glob-to-any-file: "integrations/llama_cpp/**/*"
- any-glob-to-any-file: ".github/workflows/llama_cpp.yml"

integration:ollama:
- changed-files:
- any-glob-to-any-file: "integrations/ollama/**/*"
- any-glob-to-any-file: ".github/workflows/ollama.yml"

integration:opensearch:
- changed-files:
- any-glob-to-any-file: "integrations/opensearch/**/*"
- any-glob-to-any-file: ".github/workflows/opensearch.yml"

integration:qdrant:
- changed-files:
- any-glob-to-any-file: "integrations/qdrant/**/*"
- any-glob-to-any-file: ".github/workflows/qdrant.yml"

integration:pinecone:
- changed-files:
- any-glob-to-any-file: "integrations/pinecone/**/*"
- any-glob-to-any-file: ".github/workflows/pinecone.yml"

integration:qdrant:
- changed-files:
- any-glob-to-any-file: "integrations/qdrant/**/*"
- any-glob-to-any-file: ".github/workflows/qdrant.yml"

integration:unstructured-fileconverter:
- changed-files:
- any-glob-to-any-file: "integrations/unstructured/fileconverter/**/*"
- any-glob-to-any-file: ".github/workflows/unstructured_fileconverter.yml"

integration:ollama:
integration:uptrain:
- changed-files:
- any-glob-to-any-file: "integrations/ollama/**/*"
- any-glob-to-any-file: ".github/workflows/ollama.yml"
- any-glob-to-any-file: "integrations/uptrain/**/*"
- any-glob-to-any-file: ".github/workflows/uptrain.yml"

integration:llama_cpp:
integration:weaviate:
- changed-files:
- any-glob-to-any-file: "integrations/llama_cpp/**/*"
- any-glob-to-any-file: ".github/workflows/llama_cpp.yml"
- any-glob-to-any-file: "integrations/weaviate/**/*"
- any-glob-to-any-file: ".github/workflows/weaviate.yml"

# Topics
topic:CI:
Expand Down
56 changes: 56 additions & 0 deletions .github/workflows/amazon_sagemaker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# This workflow comes from https://github.com/ofek/hatch-mypyc
# https://github.com/ofek/hatch-mypyc/blob/5a198c0ba8660494d02716cfc9d79ce4adfb1442/.github/workflows/test.yml
name: Test / amazon-sagemaker

on:
schedule:
- cron: "0 0 * * *"
pull_request:
paths:
- "integrations/amazon_sagemaker/**"
- ".github/workflows/amazon_sagemaker.yml"

defaults:
run:
working-directory: integrations/amazon_sagemaker

concurrency:
group: amazon-sagemaker-${{ github.head_ref }}
cancel-in-progress: true

env:
PYTHONUNBUFFERED: "1"
FORCE_COLOR: "1"

jobs:
run:
name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.9", "3.10"]

steps:
- name: Support longpaths
if: matrix.os == 'windows-latest'
working-directory: .
run: git config --system core.longpaths true

- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- name: Lint
if: matrix.python-version == '3.9' && runner.os == 'Linux'
run: hatch run lint:all

- name: Run tests
run: hatch run cov
60 changes: 60 additions & 0 deletions .github/workflows/astra.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# This workflow comes from https://github.com/ofek/hatch-mypyc
# https://github.com/ofek/hatch-mypyc/blob/5a198c0ba8660494d02716cfc9d79ce4adfb1442/.github/workflows/test.yml
name: Test / astra

on:
schedule:
- cron: "0 0 * * *"
pull_request:
paths:
- 'integrations/astra/**'
- '.github/workflows/astra.yml'

defaults:
run:
working-directory: integrations/astra

concurrency:
group: astra-${{ github.head_ref }}
cancel-in-progress: true

env:
PYTHONUNBUFFERED: "1"
FORCE_COLOR: "1"

jobs:
run:
name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
max-parallel: 1
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.9', '3.10']

steps:
- name: Support longpaths
if: matrix.os == 'windows-latest'
working-directory: .
run: git config --system core.longpaths true

- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- name: Lint
if: matrix.python-version == '3.9' && runner.os == 'Linux'
run: hatch run lint:all

- name: Run tests
env:
ASTRA_DB_APPLICATION_TOKEN: ${{ secrets.ASTRA_DB_APPLICATION_TOKEN }}
ASTRA_DB_ID: ${{ secrets.ASTRA_DB_ID }}
run: hatch run cov
6 changes: 5 additions & 1 deletion .github/workflows/chroma.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,9 @@ jobs:
if: matrix.python-version == '3.9' && runner.os == 'Linux'
run: hatch run lint:all

- name: Generate docs
if: matrix.python-version == '3.9' && runner.os == 'Linux'
run: hatch run docs

- name: Run tests
run: hatch run cov
run: hatch run cov
58 changes: 58 additions & 0 deletions .github/workflows/pgvector.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# This workflow comes from https://github.com/ofek/hatch-mypyc
# https://github.com/ofek/hatch-mypyc/blob/5a198c0ba8660494d02716cfc9d79ce4adfb1442/.github/workflows/test.yml
name: Test / pgvector

on:
schedule:
- cron: "0 0 * * *"
pull_request:
paths:
- "integrations/pgvector/**"
- ".github/workflows/pgvector.yml"

concurrency:
group: pgvector-${{ github.head_ref }}
cancel-in-progress: true

env:
PYTHONUNBUFFERED: "1"
FORCE_COLOR: "1"

jobs:
run:
name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9","3.10","3.11"]
services:
pgvector:
image: ankane/pgvector:latest
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- name: Lint
working-directory: integrations/pgvector
if: matrix.python-version == '3.9'
run: hatch run lint:all

- name: Run tests
working-directory: integrations/pgvector
run: hatch run cov
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# This workflow comes from https://github.com/ofek/hatch-mypyc
# https://github.com/ofek/hatch-mypyc/blob/5a198c0ba8660494d02716cfc9d79ce4adfb1442/.github/workflows/test.yml
name: Test / unstructured / fileconverter
name: Test / unstructured

on:
schedule:
- cron: "0 0 * * *"
pull_request:
paths:
- "integrations/unstructured/fileconverter/**"
- ".github/workflows/unstructured_fileconverter.yml"
- "integrations/unstructured/**"
- ".github/workflows/unstructured.yml"

concurrency:
group: unstructured_fileconverter-${{ github.head_ref }}
group: unstructured-${{ github.head_ref }}
cancel-in-progress: true

env:
Expand Down Expand Up @@ -50,10 +50,10 @@ jobs:
run: pip install --upgrade hatch

- name: Lint
working-directory: integrations/unstructured/fileconverter
working-directory: integrations/unstructured
if: matrix.python-version == '3.9'
run: hatch run lint:all

- name: Run tests
working-directory: integrations/unstructured/fileconverter
working-directory: integrations/unstructured
run: hatch run cov
Loading

0 comments on commit a999960

Please sign in to comment.