Where do I have to install the libssh-dev? on the sftp server?

Do you have an example to read a file from an sftp server with R via scp?

Hi @ambullo,

you need to install the libssh-dev into the script language container together with the ssh R package. Here is the project page of the package https://github.com/ropensci/ssh, it also contains an example how to read a file from the ssh server. SCP uses a different protocol than SFTP, so please check if you can access your server also via SCP before you attempt it from the UDFs.

For customizing script language (e.g. installing new packages), please read the following section about customizing flavors in our Readme. If attempt to customize the script language container, please also checkout the prerequisites and subsequent sections.

Let me know, if you encounter any problems.

Regards,
@tkilias

Hi tkilias

Thanks a lot for your answer. We will need to see what is the best option.

Regards,
Andreas

Hi @ambullo ,

I am going to work on this now. I let you know, when I have a prototype.

Regards,
@tkilias

Hi @tkilias

I tried now the solution in your initial post. Here are some questions I run into:

1. Remove curl from the normal package lists of the flavor
   Just in this file or any other "./flavor_base/build_deps/packages/apt_get_packages" ?

2. Install libssh before running the apt install scripts
   Is it correct to add this

RUN apt-get install -y libssh-dev

in the file "./flavor_base/build_deps/docker"

Before the command
RUN apt-get -y update && \ apt-get install -y $(cat /build_info/packages/build_deps/apt_get_packages) && \...

3. Recompile libcurl with libssh following these instructions before running the apt install scripts
   Do I have to put the code for the recompile in this docker file as well
   "./flavor_base/build_deps/docker"
   are there any other adaptions I have to make?
   (I added RUN before every command)

Regards
@ambullo

Hi @ambullo ,

I am working on a patch for the fancy-r 6.1 flavor which compile libcurl with libssh support. I am currently, figuring out which changes are necessary. From what I saw, until there are many small changes necessary. I think, I will have a prototype at the end of the day.

Regards,
@tkilias

Hi @tkilias

Okay, thanks for the update. Let me know if when you have figured it out.

Best
@ambullo

Hi @ambullo,

I finished on last Friday evening the prototype. Please, see the pull request. Unfortunately, I can't integrate in into the master, because the patch takes some questionable options for installing other Ubuntu packages and increases the build time quite a lot. You can try it and use it, but won't merge it. Let me know, if you have any problems.

@ambullo Also another question, do you really need R or can you use Python, because for Python we provide the package pysftp out of the box.

Hi @tkilias

I tried it last friday afternoon and the build ended in an error. I will try it again now and let you know if it works.
Python us currently not an option since most of the users are not familiar with python and prefere to work with R.

Hi @tkilias

It worked to create the container and it is also possible to establish the sftp connection with Rcurl on Exasol. Thanks for your work!

Can you elaborate a little bit about your statement

the patch takes some questionable options for installing other Ubuntu packages

are there potential issues when we deploy this patch on our customers server?

Hi @ambullo ,

Great.

The apt options should not have an impact on the container itself. I needed to activate --allow-downgrade and had to pin some package versions. Otherwise, apt tried to install incompatible versions. This might make problems in maintaining this flavor that is one of the reasons, I won't merge for the time being. The other reason is, that the build time of curl is quite high and this would affect all future builds. Unless, I find a solution to both problems, I will leave this feature in a separate branch.

I am closing the ticket for the moment, because I can't merge these changes easily. Feel free to reopen it, if there is dire need to have it merged.