Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump scala-libs to v32.41.0 #1124

Merged
merged 28 commits into from
Apr 16, 2024
Merged

Bump scala-libs to v32.41.0 #1124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3c75aa8
Bump scala-libs to v32.41.0
weco-bot Apr 11, 2024
bf9ac59
Try localstack3.0.0 to fix sqs failure
agnesgaroux Apr 12, 2024
f35a18b
Don't set the version
agnesgaroux Apr 12, 2024
1cfe38e
Dependencies
agnesgaroux Apr 12, 2024
cc6068d
CRT not used as default anymore?
agnesgaroux Apr 12, 2024
6c1f455
ugh
agnesgaroux Apr 12, 2024
f34fe6d
unused import
agnesgaroux Apr 12, 2024
3858e1a
take???
agnesgaroux Apr 12, 2024
405435f
what's going on
agnesgaroux Apr 12, 2024
3e4e788
dang
agnesgaroux Apr 12, 2024
0f9f5e9
revert changes
agnesgaroux Apr 12, 2024
acbc622
use localstack 3.0.0 in common
agnesgaroux Apr 12, 2024
5a30ebb
try to set region as env var
agnesgaroux Apr 12, 2024
162dbd5
The provided DefaultS3AsyncClient is not an instance of S3CrtAsyncClient
agnesgaroux Apr 12, 2024
d28d3d5
use S3AsyncClient.crtCreate
agnesgaroux Apr 12, 2024
425a115
typo
agnesgaroux Apr 12, 2024
da363b8
rollback
agnesgaroux Apr 15, 2024
f2f22d2
use crt builder in Main
agnesgaroux Apr 15, 2024
a5a1f11
use crt builder in fixtures
agnesgaroux Apr 15, 2024
6542648
azurite doesn't like v3.0.0
agnesgaroux Apr 15, 2024
a2edc8d
formatting
agnesgaroux Apr 15, 2024
79b9787
set AWS_REGION for localstack SQS
agnesgaroux Apr 15, 2024
8b24356
ALLOW_NONSTANDARD_REGIONS in services using localstack/sqs
agnesgaroux Apr 15, 2024
9c6c8bd
formatting
agnesgaroux Apr 15, 2024
b453bbd
with S3CrtAsyncClient
agnesgaroux Apr 15, 2024
238369a
use S3AsyncClient.crtBuilder()
agnesgaroux Apr 15, 2024
9b5d2d7
use no_alpine sbt-wrapper image
agnesgaroux Apr 16, 2024
ce1ed2f
revert setting region
agnesgaroux Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion bag_register/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion bag_replicator/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
9 changes: 7 additions & 2 deletions bag_replicator/src/main/scala/weco/storage_service/bag_replicator/Main.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package weco.storage_service.bag_replicator
import akka.actor.ActorSystem
import com.azure.storage.blob.{BlobServiceClient, BlobServiceClientBuilder}
import com.typesafe.config.Config
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.{S3AsyncClient, S3Client}
import software.amazon.awssdk.services.s3.presigner.S3Presigner
import software.amazon.awssdk.services.sqs.SqsAsyncClient
import software.amazon.awssdk.transfer.s3.S3TransferManager
Expand Down Expand Up @@ -52,8 +52,13 @@ object Main extends WellcomeTypesafeApp {

implicit val s3Client: S3Client =
S3Client.builder().build()

implicit val s3AsyncClient: S3AsyncClient =
S3AsyncClient.crtBuilder().build()

implicit val s3TransferManager: S3TransferManager =
S3TransferManager.builder().build()
S3TransferManager.builder().s3Client(s3AsyncClient).build()

implicit val s3Presigner: S3Presigner =
S3Presigner.builder().build()

Expand Down
2 changes: 1 addition & 1 deletion bag_replicator/src/test/scala/weco/storage/transfer/fixtures/S3TransferManagerFixtures.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import java.net.URI
trait S3TransferManagerFixtures extends S3Fixtures {
def createS3TransferManagerWithEndpoint(endpoint: String): S3TransferManager = {
val s3AsyncClient =
S3AsyncClient.builder()
S3AsyncClient.crtBuilder()
.credentialsProvider(s3Credentials)
.forcePathStyle(true)
.endpointOverride(new URI(endpoint))
Expand Down
3 changes: 2 additions & 1 deletion bag_root_finder/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion bag_tagger/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion bag_unpacker/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion bag_verifier/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
version: "2.1"
services:
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion bag_versioner/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
dynamodb:
Expand Down
2 changes: 1 addition & 1 deletion builds/run_sbt_task_in_docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,4 @@ docker run --tty --rm \
--net host \
--volume "$ROOT:$ROOT" \
--workdir "$ROOT" \
"$ECR_REGISTRY/wellcome/sbt_wrapper" "$@"
"$ECR_REGISTRY/wellcome/sbt_wrapper:no_alpine" "$@"
3 changes: 2 additions & 1 deletion common/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
version: "2.1"
services:
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
s3:
Expand Down
3 changes: 2 additions & 1 deletion indexer/bag_indexer/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ elasticsearch:
- "discovery.type=single-node"
- "xpack.security.enabled=false"
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
3 changes: 2 additions & 1 deletion indexer/file_finder/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
3 changes: 2 additions & 1 deletion indexer/file_indexer/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ elasticsearch:
- "discovery.type=single-node"
- "xpack.security.enabled=false"
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
3 changes: 2 additions & 1 deletion indexer/ingests_indexer/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ elasticsearch:
- "discovery.type=single-node"
- "xpack.security.enabled=false"
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
3 changes: 2 additions & 1 deletion ingests/ingests_worker/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
3 changes: 2 additions & 1 deletion notifier/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"
6 changes: 3 additions & 3 deletions project/Dependencies.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import sbt._

object WellcomeDependencies {
val defaultVersion = "32.40.4" // This is automatically bumped by the scala-libs release process, do not edit this line manually
val defaultVersion = "32.41.0" // This is automatically bumped by the scala-libs release process, do not edit this line manually

lazy val versions = new {
val fixtures = defaultVersion
Expand Down Expand Up @@ -102,12 +102,12 @@ object ExternalDependencies {

// This should match the version of aws used in scala-libs; see
// https://github.com/wellcomecollection/scala-libs/blob/main/project/Dependencies.scala
val aws = "2.19.0"
val aws = "2.25.28"

// These are the "Common Runtime Libraries", which you're encouraged to use for
// better performance.
// See https://docs.aws.amazon.com/sdkref/latest/guide/common-runtime.html
val awsCrt = "0.20.3"
val awsCrt = "0.29.16"
}

val azureDependencies: Seq[ModuleID] = Seq(
Expand Down
3 changes: 2 additions & 1 deletion replica_aggregator/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ dynamodb:
ports:
- "45678:8000"
localstack:
image: "760097843905.dkr.ecr.eu-west-1.amazonaws.com/localstack/localstack"
image: "public.ecr.aws/localstack/localstack:3.0.0"
environment:
- SERVICES=sqs
- ALLOW_NONSTANDARD_REGIONS=1
ports:
- "4566:4566"