Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use sftp RENAME instead of copy and delete in case of moving a file #40

Open
florian-dacosta opened this issue Dec 27, 2023 · 1 comment

Comments

@florian-dacosta
Copy link

Hello,
I use fsspec to have a single api to work with multiple different storage backend.
I ran into some issues with the sftp implementation because the move method is used and fails with the error : asyncssh.misc.ChannelOpenError: Session request failed trying to do a cp command here https://github.com/fsspec/sshfs/blob/main/sshfs/spec.py#L162

From what I understand the cp command is not always (rarely ?) possible on sftp protocol and the posix_rename is not always supported either.
Would it be possible to replace this to copy and then remove the file by a sftp rename ?
In my case, that did fix the issue and this is the approach used for example in the fs.sshfs implementation. (https://github.com/althonos/fs.sshfs/blob/master/fs/sshfs/sshfs.py#L279)
And it seems quite easy since the rename is already implemented in asyncssh.
I could do a PR about it is an acceptable change.

@efiop
Copy link
Member

efiop commented Dec 28, 2023

Hey Florian,

We are trying to cp over ssh, not over sftp, so I wonder if your server (what is it btw?) doesn't like/support us trying to run commands through ssh. It would be great to look around to understand what is specifically wrong there.

Regarding rename, I don't remember all the specifics about its differences with posix_rename from the top of my head (with exception of atomicity, IIRC), but it indeed shouldn't be worse than cp cmd that we are trying to run that is not atomic anyways, so we should indeed be able eto fallback to it one way or another (or maybe even just replace cp, I haven't thought deeply about it though). Obviously, PR is welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants