Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

destination-redshift: Use airbyte/java-connector-base:1.0.0 #49903

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

destination-redshift: Use airbyte/java-connector-base:1.0.0 #49903

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c611436
base-image-migration
alafanechere Dec 18, 2024
4e8ae3f
chore: auto-fix lint and format issues
octavia-squidington-iii Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 58 additions & 41 deletions airbyte-integrations/connectors/destination-redshift/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,46 @@ data:
ab_internal:
ql: 200
sl: 100
connectorBuildOptions:
baseImage: docker.io/airbyte/java-connector-base:1.0.0@sha256:be86e5684e1e6d9280512d3d8071b47153698fe08ad990949c8eeff02803201a
connectorSubtype: database
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- fileName: 1s1t_config.json
name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: 1s1t_config_raw_schema_override.json
name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_RAW_SCHEMA_OVERRIDE_DISABLE_TYPING_DEDUPING
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: 1s1t_config_staging.json
name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_STAGING
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: 1s1t_config_staging_raw_schema_override.json
name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_STAGING_RAW_SCHEMA_OVERRIDE_DISABLE_TYPING_DEDUPING
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: config_staging.json
name: SECRET_DESTINATION-REDSHIFT_STAGING__CREDS
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: config.json
name: SECRET_DESTINATION-REDSHIFT__CREDS
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
connectorType: destination
definitionId: f7a7d195-377f-cf5b-70a5-be6b819019dc
dockerImageTag: 3.5.0
dockerImageTag: 3.5.1
dockerRepository: airbyte/destination-redshift
documentationUrl: https://docs.airbyte.com/integrations/destinations/redshift
githubIssueLabel: destination-redshift
Expand All @@ -17,61 +53,42 @@ data:
enabled: true
oss:
enabled: true
releaseStage: generally_available
releases:
breakingChanges:
2.0.0:
message: >
This version introduces [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2), which provides better error handling, incremental delivery of data for large syncs, and improved final table structures. To review the breaking changes, and how to upgrade, see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading). These changes will likely require updates to downstream dbt / SQL models, which we walk through [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#updating-downstream-transformations).
Selecting `Upgrade` will upgrade **all** connections using this destination at their next sync. You can manually sync existing connections prior to the next scheduled sync to start the upgrade early.
message:
"This version introduces [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2),
which provides better error handling, incremental delivery of data for large
syncs, and improved final table structures. To review the breaking changes,
and how to upgrade, see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading).
These changes will likely require updates to downstream dbt / SQL models,
which we walk through [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#updating-downstream-transformations).
Selecting `Upgrade` will upgrade **all** connections using this destination
at their next sync. You can manually sync existing connections prior to
the next scheduled sync to start the upgrade early.

"
upgradeDeadline: "2024-03-15"
3.0.0:
message: >
Version 3.0.0 of destination-redshift removes support for the "standard inserts" mode. S3 staging was always preferred for being faster and less expensive, and as part of Airbyte 1.0, we are officially removing the inferior "standard inserts" mode. Upgrading to this version of the destination will require a configuration with an S3 staging area.
message:
'Version 3.0.0 of destination-redshift removes support for the "standard
inserts" mode. S3 staging was always preferred for being faster and less
expensive, and as part of Airbyte 1.0, we are officially removing the inferior
"standard inserts" mode. Upgrading to this version of the destination will
require a configuration with an S3 staging area.

'
upgradeDeadline: "2024-07-31"
resourceRequirements:
jobSpecific:
- jobType: sync
resourceRequirements:
memory_limit: 1Gi
memory_request: 1Gi
releaseStage: generally_available
supportLevel: certified
supportsDbt: true
supportsRefreshes: true
tags:
- language:java
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG
fileName: 1s1t_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_RAW_SCHEMA_OVERRIDE_DISABLE_TYPING_DEDUPING
fileName: 1s1t_config_raw_schema_override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_STAGING
fileName: 1s1t_config_staging.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-REDSHIFT_1S1T_CONFIG_STAGING_RAW_SCHEMA_OVERRIDE_DISABLE_TYPING_DEDUPING
fileName: 1s1t_config_staging_raw_schema_override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-REDSHIFT_STAGING__CREDS
fileName: config_staging.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-REDSHIFT__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
9 changes: 5 additions & 4 deletions docs/integrations/destinations/redshift.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,10 +222,11 @@ Each stream will be output into its own raw table in Redshift. Each table will c

| Version | Date | Pull Request | Subject |
| :------ | :--------- | :--------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 3.5.0 | 2024-09-18 | [45435](https://github.com/airbytehq/airbyte/pull/45435) | upgrade all dependencies |
| 3.4.4 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb |
| 3.4.3 | 2024-08-22 | [44526](https://github.com/airbytehq/airbyte/pull/44526) | Revert protocol compliance fix |
| 3.4.2 | 2024-08-15 | [42506](https://github.com/airbytehq/airbyte/pull/42506) | Fix bug in refreshes logic (already mitigated in platform, just fixing protocol compliance) |
| 3.5.1 | 2024-12-18 | [49903](https://github.com/airbytehq/airbyte/pull/49903) | Use a base image: airbyte/java-connector-base:1.0.0 |
| 3.5.0 | 2024-09-18 | [45435](https://github.com/airbytehq/airbyte/pull/45435) | upgrade all dependencies |
| 3.4.4 | 2024-08-20 | [44476](https://github.com/airbytehq/airbyte/pull/44476) | Increase message parsing limit to 100mb |
| 3.4.3 | 2024-08-22 | [44526](https://github.com/airbytehq/airbyte/pull/44526) | Revert protocol compliance fix |
| 3.4.2 | 2024-08-15 | [42506](https://github.com/airbytehq/airbyte/pull/42506) | Fix bug in refreshes logic (already mitigated in platform, just fixing protocol compliance) |
| 3.4.1 | 2024-08-13 | [xxx](https://github.com/airbytehq/airbyte/pull/xxx) | Simplify Redshift Options |
| 3.4.0 | 2024-07-23 | [42445](https://github.com/airbytehq/airbyte/pull/42445) | Respect the `drop cascade` option on raw tables |
| 3.3.1 | 2024-07-15 | [41968](https://github.com/airbytehq/airbyte/pull/41968) | Don't hang forever on empty stream list; shorten error message on INCOMPLETE stream status |
Expand Down
Loading