Skip to content

Commit

Permalink
adjusting intro pages
Browse files Browse the repository at this point in the history
  • Loading branch information
stmorse committed Dec 13, 2024
1 parent 85b2c89 commit 4a7ffa4
Show file tree
Hide file tree
Showing 4 changed files with 66 additions and 5 deletions.
2 changes: 1 addition & 1 deletion _data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
- title: 👋 Login & Basic Setup
url: /hpc-gitbook/logging-in-and-setting-up-your-hpc-account/login-and-basic-setup.html
- title: 📁 Uploading Files
url: /hpc-gitbook/logging-in-and-setting-up-your-hpc-account/filezilla.html
url: /hpc-gitbook/logging-in-and-setting-up-your-hpc-account/moving-files.html
- title: 🔒 Using the Jump Server and Configuring SSH
url: /hpc-gitbook/logging-in-and-setting-up-your-hpc-account/configuring-ssh.html

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Using the clusters from a personal laptop not on the WM network requires you to
In this section, we'll summarize how to do this step, and then how to make our SSH lives easier by using SSH keys and setting up `~/.ssh/config`.


## Using `bastion`
## Using the Jump Server (bastion)

WM isolates its clusters on an internal private network and controls outside access using a "bastion" host, which acts as a sort of gatekeeper between the outside world and the internal school network. (The term *bastion* originally referred to a critical part of a fortification.)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,20 @@

## How to get your files on the HPC

Just like SSH provides us a secure *shell* for working with the cluster, we also need a way to secure *copy* or move files.
Just like SSH provides us a secure *shell* for working with the cluster, we also need a way to secure *copy* or move files from one machine to another (for example, your local computer to the cluster). We'll look at three approaches:

- **Using command line tools like `scp` or `sftp`.** This is probably the most reliable method and the best choice for large files, but is strictly for file transfer and can be cumbersome for complicated work.

- **Using a GUI like Filezilla.** This is essentially a wrapper of `sftp`, and makes file transfer a little more user-friendly, but is still strictly for file transfer and probably not a good choice for large files.

- **Mounting a drive using `sshfs`.** This is the most flexible option, since it allows moving and transferring files with your local, OS-native file explorer, *and* opening/editing files as if they were on your local machine. Still not a great choice for moving large files (due to the risk of mid-transfer disconnect or syncing issues) but a good option for editing scripts --- we cover mounting a drive in a [separate post here](https://d8a-science.github.io/hpc-gitbook/logging-in-and-setting-up-your-hpc-account/using-sshfs.html).


## Using the command line

The most straightforward and out-of-the-box way is using the command line tool [`scp`](https://haydenjames.io/linux-securely-copy-files-using-scp/). Let's say you're on your local machine and want to move a file in your current working directory to the subfolder `test` in your home directory on the cluster. You can do (for example):

```
```bash
scp myFile.txt <user>@bora.sciclone.wm.edu:test/
```

Expand All @@ -25,7 +34,7 @@ While not appropriate for large files (say, >100Gb), there are also nice GUI opt

FileZilla is predominantly a GUI-based way to access FTP and SFTP sites, and as such has a simple graphical installer you can download for any platform here: [https://filezilla-project.org/](https://filezilla-project.org/)

## Basic Use
### Basic Use

<!-- ![step1](https://user-images.githubusercontent.com/7882645/190205224-068f3893-07a9-4bf5-8165-9e61e248266e.png) -->

Expand Down
52 changes: 52 additions & 0 deletions logging-in-and-setting-up-your-hpc-account/using-sshfs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Mounting a remote drive locally

You will likely find it convenient to "mount" a remote drive (like your `/sciclone/home/<user>` directory) on your local, personal computer. This means you add the directory to your local file system in a way that looks just like a local directory to your computer and all the apps on it. Once you do this, you can open your remote folder in your Files/File Explorer/Finder app just like any other folder, move files around, etc -- you can even open scripts on the remote machine in a coding IDE on your local computer, without any plugins/SSH necessary!

Let's learn how to do this. It is easy on Linux, and fairly easy on Mac and Windows. Instructions for Windows are not included here (if you have done it, send a PR to this repo and we'll add it!)


## Mounting on Linux

For Linux, we'll use the [`sshfs` utility](https://en.wikipedia.org/wiki/SSHFS) ("SSH Filesystem"). First, make sure it's installed:

```bash
sudo apt update
sudo apt install sshfs
```

Then create a directory where the remote directory will be mounted, for example:

```bash
cd ~
mkdir mounts
cd mounts
mkdir sciclone
```

Now, use `sshfs` to mount the remote directory. Here we'll mount our home directory on SciClone (this uses the shorthand `bora` because we're assuming you've already setup your [SSH config file](https://d8a-science.github.io/hpc-gitbook/logging-in-and-setting-up-your-hpc-account/configuring-ssh.html), if not you'll need to use the full `<user>@bora.sciclone.wm.edu`):

```bash
sshfs bora:/sciclone/home/<your-user-name> ~/mounts/sciclone
```

If successful, it will return silently (no output) and you can try:

```bash
cd ~/mounts/sciclone
```

You should see your folders. Try making a file with `touch test.txt`. Now SSH directly into the remote server and you'll see the file is there. Yay!

## Mounting at startup

If you restart your computer, it will disconnect this mount and you'll still see your `~/mounts/sciclone` directory but it'll be empty. Instead of manually reconnecting on startup every time, there are several ways to do this automatically on startup, that we will not cover in detail here but you can look up:

- Add a line to your `/etc/fstab` file ("filesystems table") which tells Linux how to mount drives on startup.
- Create a bash script `mount.sh` with the sshfs command and then configure Linux to run this script on startup ("Startup Applications")
- Use `systemd` to manage the mount (this is the most robust solution) but also most involved.

And another option is to just rarely restart your computer and

## Technical notes

-

0 comments on commit 4a7ffa4

Please sign in to comment.