Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fs.onedatarestfs for Onedata files source plugin implementation #16690

Merged
merged 4 commits into from
Sep 29, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions lib/galaxy/config/sample/file_sources_conf.yml.sample
Original file line number Diff line number Diff line change
Expand Up @@ -197,3 +197,10 @@
doc: Invenio RDM turn-key research data management repository
label: Invenio RDM Demo Repository
url: https://inveniordm.web.cern.ch/

- type: onedata
id: onedata1
label: Onedata
doc: Your Onedata files - configure an access token via the user preferences
lopiola marked this conversation as resolved.
Show resolved Hide resolved
accessToken: ${user.preferences['onedata|access_token']}
onezoneDomain: ${user.preferences['onedata|onezone_domain']}
13 changes: 13 additions & 0 deletions lib/galaxy/config/sample/user_preferences_extra_conf.yml.sample
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,16 @@ preferences:
label: Whether to make draft records publicly available or restricted.
type: boolean
required: False

# Used in file_sources_conf.yml
onedata:
description: Your Onedata account
inputs:
- name: onezone_domain
label: Domain of the Onezone service (e.g. "demo.onedata.org")
type: text
required: False
- name: access_token
label: Your access token, suitable for REST API access in a Oneprovider service
type: password
required: False
2 changes: 1 addition & 1 deletion lib/galaxy/dependencies/conditional-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ fs.googledrivefs # type: googledrive
fs-gcsfs # type: googlecloudstorage
# fs-gcsfs doesn't pin google-cloud-storage, and old versions log noisy exceptions and break test discovery
google-cloud-storage>=2.8.0 # type: googlecloudstorage
fs-onedatafs # type: onedata
fs.onedatarestfs # type: onedata
fs-basespace # type: basespace

# Vault backend
Expand Down
29 changes: 12 additions & 17 deletions lib/galaxy/files/sources/onedata.py
Original file line number Diff line number Diff line change
@@ -1,30 +1,25 @@
try:
from fs.onedatafs import OnedataFS
from fs.onedatarestfs import OnedataRESTFS
except ImportError:
OnedataFS = None
OnedataRESTFS = None

from typing import (
Optional,
Union,
)

from . import (
FilesSourceOptions,
FilesSourceProperties,
)
from typing import Optional
from . import FilesSourceOptions
from ._pyfilesystem2 import PyFilesystem2FilesSource


class OneDataFilesSource(PyFilesystem2FilesSource):
class OnedataFilesSource(PyFilesystem2FilesSource):
plugin_type = "onedata"
required_module = OnedataFS
required_package = "fs-onedatafs"
required_module = OnedataRESTFS
required_package = "fs.onedatarestfs"

def _open_fs(self, user_context=None, opts: Optional[FilesSourceOptions] = None):
props = self._serialization_props(user_context)
extra_props: Union[FilesSourceProperties, dict] = opts.extra_props or {} if opts else {}
handle = OnedataFS(**{**props, **extra_props})
onezone_domain = props.pop("onezoneDomain", {}) or ""
lopiola marked this conversation as resolved.
Show resolved Hide resolved
onezone_domain = onezone_domain.lstrip("https://").lstrip("http://")
access_token = props.pop("accessToken", {}) or ""
lopiola marked this conversation as resolved.
Show resolved Hide resolved
handle = OnedataRESTFS(onezone_domain, access_token)
return handle


__all__ = ("OneDataFilesSource",)
__all__ = ("OnedataFilesSource",)