Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added detailed guide on how to set up SSH for the HPC cluster #41

Merged
merged 9 commits into from
Oct 3, 2023

Conversation

niksirbi
Copy link
Member

@niksirbi niksirbi commented Sep 27, 2023

I added a detailed guide on how to SSH into the SWC's HPC cluster.
I also included sections on how to configure the .ssh/config file and how to set up SSH key pair authentication.
These set up steps should make future interactions with the cluster smoother and will aid remote development.

The text contains many Sphinx design elements and is not easily readable in raw markdown form.
I suggest locally building the website to review the rendered result:

  • Clone the repo and check out this branch
  • Crate a new coding environment (conda or otherwise) and activate it
  • Install all the build requirements via pip install -r docs/requirements.txt
  • Build the website locally via sphinx-build docs/source docs/build (if you build multiple times, make sure to remove the old build via rm -rf docs/build)
  • The rendered website is available at docs/build/index.html

As reviewers I'm tagging:

  • @JoeZiminski because he is a Windows user and I'm not entirely sure this works on Windows
  • @IgorTatarnikov because he is new at SWC and probably needs to do something like this anyway.

@niksirbi niksirbi changed the title add resusable snippets on code blocks and SWC wiki Added detailed guide on how to set up SSH for the HPC cluster Sep 27, 2023
Copy link
Member

@IgorTatarnikov IgorTatarnikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy to follow and worked fine on macOS while connected to the UCL VPN.

For my own reference do we have to be on the UCL/SWC VPN to access the HPC?

docs/source/programming/SSH-SWC-cluster.md Outdated Show resolved Hide resolved
docs/source/programming/SSH-SWC-cluster.md Outdated Show resolved Hide resolved
@niksirbi
Copy link
Member Author

For my own reference do we have to be on the UCL/SWC VPN to access the HPC?

I think not? I suppose the bastion node is exposed to external networks, so since we SSH through that, VPN should be unnecessary. But I should test this when home.

But you definitely need VPN to map/mount SWC filesystems on your local computer.

@niksirbi
Copy link
Member Author

Easy to follow and worked fine on macOS while connected to the UCL VPN.

Thank you @IgorTatarnikov !!

@adamltyson
Copy link
Member

I think not? I suppose the bastion node is exposed to external networks, so since we SSH through that, VPN should be unnecessary. But I should test this when home.

Yeah, this works without VPN.

Copy link
Member

@JoeZiminski JoeZiminski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @niksirbi

Looks great! very easy to follow and I'm chuffed I now have HPC keys setup on this PC, I always CBA'd doing it again but this guide made it very easy, also those aliases are very useful.

Since Windows 10, ssh comes installed, and it is possible to follow the first half of the guide at least using the file browser only. However, for the entire tutorial it is worth them using Git Bash, as windows does not have many of the useful commands (e.g. ssh-copy-id) and it is difficult to follow along with the linux syntax otherwise.

As such it might be worth having a optional drop-down box called "A note for Windows users` that reads something like the below (however, this became more complex that I expected so might not be worth it):

Since Windows 10, `ssh` is available in the command window and can be used to connect to the HPC. If you are following along with this guide to the section SSH config file', you do not need Git Bash (although, you can use it if you want to follow along with the commands in the huide) but will be able to log in using the file browser and `cmd` tool. However, for the section "SSH Keys", it is recommended to use Git Bash as this will allow you to follow along with the shown commands, and Git Bash also includes some useful functions used in this tutorial that are not provided in Windows.

In Windows `cmd`, you can use 

`ssh <SWC-USERNAME>@ssh.swc.ucl.ac.uk`
`ssh hpc-gw1`

as below natively.

However, for setting up the config file, the easist way will be to navigate using the file browser. The `.ssh` should go in your home directory, i.e. `C:\Users\<username>.ssh`. If the `.ssh` folder does not already exists, you can make it (don't forget the `.` at the start).

Then, you can create a file called 'config' using notepad. However, this file should not have an extension. To create a file without an extension in windows, you need to have the below setting switched on:

- In the file browser, click 'View'
- Check the box 'File name extensions'

Now, when you create a file called 'config' in `.ssh`, using NotePad, you can delete the extension.

You can copy and paste the information as in the below guide using NotePad, as the text editor `nano` is not available on Windows.

This is kind of wordy so it might be worth not bothering and telling them to use Git Bash, but just mentioning for day-to-day use cmd does support ssh (at least, the basic ssh command)

docs/source/programming/SSH-SWC-cluster.md Show resolved Hide resolved
docs/source/programming/SSH-SWC-cluster.md Outdated Show resolved Hide resolved
docs/source/programming/SSH-SWC-cluster.md Outdated Show resolved Hide resolved
@niksirbi niksirbi requested a review from JoeZiminski October 3, 2023 10:56
@niksirbi
Copy link
Member Author

niksirbi commented Oct 3, 2023

Hey @JoeZiminski thanks a lot for the Windows insight. I've added the dropdown at the top, giving Windows users two options:

  1. Follow along with all commands with Git Bash
  2. Use cmd, file browser, and notepad (with the caveats you mentioned)

Have another quick look and approve it if you are happy.

@JoeZiminski
Copy link
Member

JoeZiminski commented Oct 3, 2023

Looks great @niksirbi love the drop-down. Two small suggestions otherwise LGTM!

@niksirbi
Copy link
Member Author

niksirbi commented Oct 3, 2023

Thanks both @JoeZiminski and @IgorTatarnikov! I'm going to merge this now.

@niksirbi niksirbi merged commit bfc7557 into main Oct 3, 2023
3 checks passed
@niksirbi niksirbi deleted the ssh-cluster branch October 3, 2023 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants