HttpProxy and S3 as data source

[stoelmir]

Greetings,

we were using EDC version 0.2.1 and upgraded now to version 0.7.0. In version 0.2.1 it was possible to transfer an asset which data are located on a S3 bucket via HttpProxy destination type. We used this mechanism since our S3 is not accessible via public internet and only our provider can access it.

In version 0.7.0 this approach is failing now. Within the logs of the provider control plane we could find the following message: Fatal error while Initiate data flow. Error details: Transfer request failed with status code 400 for request
After a first analyzation it looks like that class PublicEndpointGeneratorServiceImpl prevents such a transfer because it does not have a public endpoint generator function for a source data address of type AmazonS3.

Do you have any proposals how to get this to work again on a easy way? We currently see some workarounds:

1. Custom extension which registers the public endpoint also for AmazonS3
2. Own http endpoint which calls the S3 bucket. The datasource could be changed then to HttpData

[paullatzelsperger]

@stoelmir Version 0.7.0 is quite old, at least in the world of EDC, so I recommend trying this with version 0.8.1 or maybe even 0.8.2-SNAPSHOT. We made some changes there recently, which should fix this.