diff --git a/docs/source/data_management/filestore.rst b/docs/source/data_management/filestore.rst index 3a02b29..b50b43a 100644 --- a/docs/source/data_management/filestore.rst +++ b/docs/source/data_management/filestore.rst @@ -20,6 +20,11 @@ To move your data from Viking to the filestore ("Storage"), the address you need You can use ``PuTTY`` on Windows and the command line on Linux and MacOS with commands such as ``rsync`` and ``scp`` much like the examples on the :ref:`Data Management page `, examples below. +.. tip:: + + When transferring large amounts of data it can be a good idea to use :doc:`terminal multiplexing ` such as ``tmux``. This would allow you to leave the transfer running (inside a ``tmux`` session) and not have to stay logged in yourself. + + Using the ``scp`` command ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -57,9 +62,9 @@ Replace ```` with the name of your file and ```` with the p .. code-block:: console - $ rsync -P --append sftp.york.ac.uk:/shared/storage/ + $ rsync -av sftp.york.ac.uk:/shared/storage/ -Option ``-P`` sets the ``--partial`` and ``--progress`` options, which will *keep partially transferred files* and shows *progress*. The ``--append`` option will *append data onto shorter files*. Together this is handy if a large file needs to be transferred but for some reason could get interrupted. Rerunning this command should pick up where it left off. +Options ``a`` set up a number of options useful for *archiving*, ``v`` for *verbose* so you can monitor the process. To copy a directory to your filestore """"""""""""""""""""""""""""""""""""" @@ -121,7 +126,7 @@ Replace ```` with the path to your filestore and ```` with .. code-block:: console - $ rsync -P --append sftp.york.ac.uk:/shared/storage// . + $ rsync -av sftp.york.ac.uk:/shared/storage// . To copy a directory from your filestore diff --git a/docs/source/data_management/other_locations.rst b/docs/source/data_management/other_locations.rst index 0bb59e2..73ce05f 100644 --- a/docs/source/data_management/other_locations.rst +++ b/docs/source/data_management/other_locations.rst @@ -34,6 +34,9 @@ You can copy your data from any Linux device to Viking using the following comma For example, you can run the following commands from a terminal running on your device to move files to Viking. +.. tip:: + + When transferring large amounts of data it can be a good idea to use :doc:`terminal multiplexing ` such as ``tmux``. This would allow you to leave the transfer running (inside a ``tmux`` session) and not have to stay logged in yourself. scp """ @@ -74,9 +77,9 @@ This is recommended for a large number of files. Rsync can check what is already $ rsync -av dirname viking.york.ac.uk:~/scratch .. code-block:: console - :caption: this can be useful for copying a very large file from your device to your scratch area on Viking as it will allow you continue the transfer if the connection breaks for some reason + :caption: this could be useful for copying a very large file from your device to your scratch area on Viking as it will allow you continue the transfer if the connection breaks for some reason - $ rsync -P --append filename viking.york.ac.uk:~/scratch + $ rsync -avP --append filename viking.york.ac.uk:~/scratch There are many more options you can use with ``rsync``. To view these options run the following command to view the ``rsync`` manual