Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

destination-databricks: Use airbyte/java-connector-base:1.0.0 #49898

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,9 +1,44 @@
data:
ab_internal:
ql: 100
sl: 100
connectorBuildOptions:
baseImage: docker.io/airbyte/java-connector-base:1.0.0@sha256:be86e5684e1e6d9280512d3d8071b47153698fe08ad990949c8eeff02803201a
connectorSubtype: database
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- fileName: azure_config.json
name: SECRET_DESTINATION-DATABRICKS_AZURE__CREDS
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: config.json
name: SECRET_DESTINATION-DATABRICKS__CREDS
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: staging_config.json
name: SECRET_DESTINATION-DATABRICKS__STAGING_CREDS
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: oauth_config.json
name: SECRET_DESTINATION_DATABRICKS_OAUTH_CONFIG
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
- fileName: pat_config.json
name: SECRET_DESTINATION_DATABRICKS_PERSONAL_ACCESS_TOKEN_CONFIG
secretStore:
alias: airbyte-connector-testing-secret-store
type: GSM
connectorType: destination
definitionId: 072d5540-f236-4294-ba7c-ade8fd918496
dockerImageTag: 3.3.1
dockerImageTag: 3.3.2
dockerRepository: airbyte/destination-databricks
documentationUrl: https://docs.airbyte.com/integrations/destinations/databricks
githubIssueLabel: destination-databricks
icon: databricks.svg
license: ELv2
Expand All @@ -17,59 +52,34 @@ data:
releases:
breakingChanges:
2.0.0:
message: >
**This is a private preview version, Do not upgrade until you review the changes**.\n
This version is a rewrite of the community connector with support for Unity catalog, and staging files using Unity catalog volumes.
message:
'**This is a private preview version, Do not upgrade until you review
the changes**.\n This version is a rewrite of the community connector with
support for Unity catalog, and staging files using Unity catalog volumes.
This version also introduces [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2),
which provides better error handling, incremental delivery of data for large syncs, and improved final table structures.
To review the breaking changes, see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading).
which provides better error handling, incremental delivery of data for large
syncs, and improved final table structures. To review the breaking changes,
see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading).

'
upgradeDeadline: "2025-01-31"
3.0.0:
message: >
This version adds an `_airbyte_generation_id` column to the raw and final tables. If you ran a sync using 2.0.0, you MUST manually drop the
raw and final tables and then clear (reset) your connection; this release will not automatically upgrade your tables.
message:
"This version adds an `_airbyte_generation_id` column to the raw
and final tables. If you ran a sync using 2.0.0, you MUST manually drop
the raw and final tables and then clear (reset) your connection; this release
will not automatically upgrade your tables.

"
upgradeDeadline: "2025-01-31"
documentationUrl: https://docs.airbyte.com/integrations/destinations/databricks
resourceRequirements:
jobSpecific:
- jobType: check_connection
resourceRequirements:
memory_limit: 600Mi
memory_request: 600Mi
tags:
- language:java
ab_internal:
sl: 100
ql: 100
supportLevel: certified
supportsRefreshes: true
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-DATABRICKS_AZURE__CREDS
fileName: azure_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-DATABRICKS__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-DATABRICKS__STAGING_CREDS
fileName: staging_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION_DATABRICKS_OAUTH_CONFIG
fileName: oauth_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION_DATABRICKS_PERSONAL_ACCESS_TOKEN_CONFIG
fileName: pat_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
tags:
- language:java
metadataSpecVersion: "1.0"
1 change: 1 addition & 0 deletions docs/integrations/destinations/databricks.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ with the raw tables, and their format is subject to change without notice.

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:--------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 3.3.2 | 2024-12-18 | [49898](https://github.com/airbytehq/airbyte/pull/49898) | Use a base image: airbyte/java-connector-base:1.0.0 |
| 3.3.1 | 2024-12-02 | [#48779](https://github.com/airbytehq/airbyte/pull/48779) | bump resource reqs for `check` |
| 3.3.0 | 2024-09-18 | [#45438](https://github.com/airbytehq/airbyte/pull/45438) | upgrade all dependencies. |
| 3.2.5 | 2024-09-12 | [#45439](https://github.com/airbytehq/airbyte/pull/45439) | Move to integrations section. |
Expand Down
Loading