Add logical object renaming feature #71
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a new POST /path/to/new/object handler takes a JSON command
description:
{
"command": "rename_from",
"source_name": "/path/to/orig/object",
"versions": [ "version1", "version2", ..],
"copy_acls": true
}
and mirrors version records under the new object name, implemented
as a logical rename (no bulk data copy/movement). The versions
field is optional and processes all versions by default. The
copy_acls field is optional and defaults to false, where only
basic ownership is set as in normal version uploads.
The implementation uses version.aux fields:
- rename_to: [ newname, newversion ]
- name: storage_name
- version: storage_version
so that the old record links to the new one and then the new
one is processed to serve content. The new record has overrides
as needed to address bulk storage via the old info in the backend.
To be more future-proof, distinguish 'hname' and 'hversion' aux values that represent overrides of URL content from the existing 'version' aux value that overrides backend S3 version addressing. This way, we do not depend on an implementation characteristic where the version string embedded in URLs is passed through to the backend version parameter. This allows for the backend strategy to change without breaking the rename logic that happens higher in the stack.
it seems like typing.NamedTuple in some Python versions cannot tolerate methods alongside the basic typed attributes.
using older (deprecated?) syntax needed for Python 3.8
mikedarcy
approved these changes
Apr 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
a new POST /path/to/new/object handler takes a JSON command description:
and mirrors version records under the new object name, implemented as a logical rename (no bulk data copy/movement). The
versionsfield is optional and it processes all versions by default. Thecopy_aclsfield is optional and defaults tofalse, where only basic ownership is set as in normal version uploads.The implementation uses
version.auxfields:rename_to: [ newname, newversion ]hname: original hatrac object name under which data is storedhversion: original hatrac object version under which data is storedThe old record links to the new one and only the new one is processed to serve content. During retrieval, the new record passes the original hatrac name information and aux metadata into the backend storage module.