Skip to content

Commit

Permalink
expand docs
Browse files Browse the repository at this point in the history
  • Loading branch information
CodyCBakerPhD committed Jul 1, 2024
1 parent 758a492 commit f8508ca
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 4 deletions.
1 change: 1 addition & 0 deletions docs/developer_guide/docker_images.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ and can then run the entrypoint (equivalent to the usual command line usage) on
.. _developer_docker_details:

Run Docker container on YAML conversion specification environment variable
--------------------------------------------------------------------------
Expand Down
23 changes: 22 additions & 1 deletion docs/user_guide/docker_demo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,4 +123,25 @@ Voilà! If everything occurred successfully, you should see...
RClone With Config Docker Demo
------------------------------

# TODO
NeuroConv also supports a convenient Docker image for running data transfers via `Rclone <https://rclone.org>`_.

To use this image, you must first configure the remote locally by calling:

.. code::
rclone config
And following all interactive instructions (defaults are usually sufficient).

The Docker image requires two environment variables to be set (see :ref:`developer_docker_details` for more details in a related process).

- ``RCLONE_CONFIG``: The full file content of the rclone.conf file on your system. You can find this by calling ``rclone config file``. On UNIX, for example, you can set this variable using ``RCLONE_CONFIG=$(<rclone.conf)`` from the folder containing the file
- ``RCLONE_COMMAND``: The Rclone command to run. For example, ``remote_name:source_folder destination_folder --verbose --progress --config ./rclone.conf``, where ``remote_name`` is the name used during initial setup through ``rclone config``, ``source_folder`` is the name of the folder you wish to transfer data from on that remote, and ``destination_folder`` is the local folder to transfer the data to.

Then, you can use the following command to run the Rclone Docker image:

.. code::
docker run -t --volume destination_folder:destination_folder -e RCLONE_CONFIG="$RCLONE_CONFIG" -e RCLONE_COMMAND="$RCLONE_COMMAND" ghcr.io/catalystneuro/rclone_with_config:latest
This image is particularly designed for convenience with AWS Batch (EC2) tools that rely heavily on atomic Docker operations. Alternative AWS approaches would have relied on transferring the Rclone configuration file to the EC2 instances using separate transfer protocols or dependent steps, both of which add complexity to the workflow.
3 changes: 0 additions & 3 deletions tests/docker_rclone_with_config_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,3 @@ def test_direct_usage_of_rclone_with_config(self):
assert (
file_content == "This is a test file for the Rclone (with config) docker image hosted on NeuroConv!"
), "The file content does not match expectations!"

def test_helper_function(self):
pass # TODO

0 comments on commit f8508ca

Please sign in to comment.