Welcome to hpcinfo!

HPC at LSHTM

Our original HPC was delivered to the school at the end of summer 2008. It was purchased from StreamLine Computing with SRIF funding, to provide a central HPC resource for research at the School.

It has been upgraded over the years with new storage and compute nodes, but the scheduler and most of the software stack have remained the same.

As we’ve now reached 2023, the HPC world is signifcantly different and much of the software/structure is now out of date.

Therefore the new HPC has been upgraded to a new operating system, Rocky Linux 8, and we have switched to a new scheduler SLURM.

The new HPC now consists of:

14 Compute nodes with between 20 and 64 cores, and 320GB to 1024GB RAM, and 1 compute node with 2 Nvidia A40 GPUs

You can find the wiki for the old HPC at http://wiki.lshtm.ac.uk/hpc/index.php5/Main_Page

Contents

• Getting Started
  • Requesting Access
  • Signing In
  • Good Practice
  • Linux Skills
  • File Access
  • Submitting Jobs
• Pryor (SSH Gateway)
• Submitting Jobs
  • Single-Threaded / Serial Job
  • Parallel Job
  • Multi-Threaded SMP Job
  • Queue Defaults
• Modules
  • How to use Modules
  • Using modules in your scripts
• Slurm Array Jobs
  • Submitting a Job Array
  • Limiting the number of simultaneous tasks
  • The $SLURM_ARRAY_TASK_ID VARIABLE
  • Output and Error files
  • Job Emails
• SLURM
  • Overview
  • Architecture
• Singularity - Docker on the HPC
  • What is a container?
  • Accessing Containers
  • Running a Container
  • Singularity and Slurm
  • Where to build Singularity Images
  • How to build Singularity images
• Conda
  • What is conda
  • Installing Conda (MiniConda)
  • Using Conda
  • Base Enviroment
  • Using conda with sbatch
  • Troubleshooting
  • Documentation
• Conda Cheet Sheet
  • Quick Start
  • Channels and packages
  • Working with conda enviroments
  • Enviroment Management
  • Exporting Enviroments
  • Importing Enviroments
  • Additional hints
• Linux Commands
  • Files and Directories
  • Getting help/documentation about a command
  • File commands (viewing/editing/creating)
  • Finding/locating Files and Applications
• Enviroment Variables
  • General
• Applications
  • Stata
  • Running R on the Cluster
  • AlphaFold

Note

This project is under development