Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AWS SDK's sts dependency to relevant modules to fix IRSA #279

Closed

Conversation

sidharrth2002
Copy link
Contributor

@sidharrth2002 sidharrth2002 commented Oct 9, 2023

What

Add software.amazon.awssdk:sts dependency to potentially resolve error caused by AWS v2 SDK.

The relevant error here was raised by @larrywax, relating to using IRSA credentials. Namely, this line was in the error log:

WebIdentityTokenCredentialsProvider(): To use web identity tokens, the 'sts' service module must be on the class path.,

How

Airbyte's use of the AWS v2 SDK for S3 data retrieval might require adding software.amazon.awssdk:sts due to a hardcoded class path in AWS's WebIdentityTokenCredentialsProvider causing errors.

Recommended reading order

  1. deps.toml
  2. airbyte-commons-worker/build.gradle
  3. airbyte-config/config-models/build.gradle
  4. airbyte-server/build.gradle
  5. airbyte-workers/build.gradle

Can this PR be safely reverted / rolled back?

If you know that your PR is backwards-compatible and can be simply reverted or rolled back, check the YES box.

Otherwise if your PR has a breaking change, like a database migration for example, check the NO box.

If unsure, leave it blank.

  • YES 💚
  • NO ❌

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Copy link

@weichunnn weichunnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bringing up the changes to resolve the missing STS module!

airbyte-oauth/build.gradle Outdated Show resolved Hide resolved
@sidharrth2002 sidharrth2002 marked this pull request as ready for review October 11, 2023 03:27
@sidharrth2002 sidharrth2002 changed the title [DRAFT] add sts dependency to relevant modules Add AWS SDK's sts dependency to relevant modules to fix IRSA Oct 11, 2023
@weichunnn
Copy link

Found other modules which are relying on implementation libs.s3 here. I think we can include the STS module there as well to have better coverage. What do you think?

@sidharrth2002
Copy link
Contributor Author

Agreed @weichunnn, I've added this as well to airbyte-commons-server and airbyte-container-orchestrator

@pmossman
Copy link
Contributor

Thanks for the continued contributions in this area @weichunnn and @sidharrth2002! I'll open this PR in our internal repo and run tests there.

@pmossman
Copy link
Contributor

/create-oss-pr

@pmossman
Copy link
Contributor

@sidharrth2002 @weichunnn sorry for the delay here - the internal build was failing and I didn't have time to investigate until today. Turns out there was a small issue with our PR-cloning tool and the fix was simple, so I went ahead and merged the internal PR. The changes will be reflected back to this public repo, and the fix will be available in the next OSS release. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants