-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add move to object client + change datasets.mv #163
base: master
Are you sure you want to change the base?
Conversation
Can this and #162 combined maybe? Or either way have to keep that other one in mind for this change too, right? |
Yeah. Either combine or I'll merge once #162 hits master. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, left a few comments.
An important thing to note is that this move endpoint works asynchronously - the lifetime of the request looks like:
- Receive request
- List all files under the source path
- Work out all the above files' destination paths and dispatch individual move operations to a queue
- Request completes
- The service churns through the queue of operations to complete
At the very least, the fact that this operation is asynchronous should be well documented. We should also consider whether it is acceptable to change the behaviour of datasets.mv
- users of this interface may rely on it blocking until complete in scripts etc.
Move object from this source path. | ||
destination : str | ||
Move object to this destination path. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: remove extra blank line
if err.error_code == "source_path_not_found": | ||
raise PathNotFound(source) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should re-raise the original exception if the error code does not match:
if err.error_code == "source_path_not_found": | |
raise PathNotFound(source) | |
if err.error_code == "source_path_not_found": | |
raise PathNotFound(source) | |
else: | |
raise |
@@ -378,6 +378,28 @@ def test_object_client_copy_source_is_a_directory(mocker): | |||
client.copy(PROJECT_ID, "source", "destination") | |||
|
|||
|
|||
def test_object_client_move_url_encoding(mocker): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this also functions as the main test of ObjectClient.move
, I suggest naming this:
def test_object_client_move_url_encoding(mocker): | |
def test_object_client_move(mocker): |
This PR adds move to the object client, using the
object-move
endpoint.This reduces the number of API calls for
datasets.mv
. I've tested the functionality and it seems to work as expected.