Welcome to Research Computing at Mines documentation!

General Information

• Overview of Research Computing at Mines
  • High Performance Computing (HPC)
  • Getting more help
• Policies
  • HPC & Storage Rates and Best Practices
  • Data Policy
  • HPC Etiquette
  • Login/Management Node
  • Scratch vs Home Directory
  • Home Directory Policy
  • Slurm
  • High-Performance Computing (HPC) Node Life Cycle
• Computing Options at Mines
  • Pay-per-use HPC
  • Free or Credit-based HPC Options
• Research Computing Consultation
  • Parallel Computing Consultation Guidance
• Frequently Asked Questions
  • What is HPC?
  • How do I acknowledge usage of HPC@Mines resources?
  • I’d like to install a Python library. How do I do that?
• Workshops
  • Fall 2024
  • Spring 2024
  • Past Workshops
• Publications using Mines HPC Systems

Scientific Visualization

• Graphing using Matplotlib and Creating Interactive Plots and Animations
  • Using Jupyter Lab to create a dataset and pandas dataframe
    • Introduction to 2d Plotting
    • Turn the plot into function to create a series of plots
    • Create a animation
• Paraview Server Startup and Connection Guide
  • Using ParaView by Connecting to a Server running on our HPC platform at Mines
    • Step 1: Creating a Job Submit Script and Submit
    • Step 2: What node is ParaView running on and setting up your tunnel
    • Step 2: Create a secure tunnel to the host from your client machine
    • Step 3: Start ParaView and connect to the server
    • Step 4: Adding the Server Configuration
    • Step 5: Connect to the Selected Server
    • Successful connection from ParaView client to a Remote Server on Mines’ HPC
  • Closing Paraview
• Ansys Remote Visualization Client Startup Guide
  • Using ANSYS Remote Visualization Client at Mines
    • Step 1: Submit a Slurm Job
    • Step 1: Alternative Startup using a Journal File
      • Submit a Fluent job using predefined commands
    • Step 2: Get the connection infomation and start a tunnel
      • Starting a Tunnel to the compute node host
    • Step 3: Start your local client and connect
• Jupyter Lab on HPC platforms at Mines
  • Required Module for python
  • Compute Node Jupyter Lab
  • Starting a Tunnel in a new terminal
  • Connecting to the Jupyter Lab server
  • Stopping a Jupyter Lab
    • Advanced Stopping a Jupyter Lab
  • Advanced Startup of a Jupyter Lab Server
• Remote VNC Setup Connection to Access Linux Desktop Apps
  • OnDemand Users - Linux Desktop and GUI apps
  • Basic setup using you own VNC connection app
    • Software Requirements
      • Mines’ HPC Software Modules
      • Mines’ HPC VNC Server Module
    • Step 1: Start an interactive SLURM Job
    • Step 2: Start a VNCserver:
    • Step 3: Identify where the VNCserver started and screen number
      • Some useful commands:
    • Step 4: Setting up a SSH Tunnel
    • Step 5: Start the VNC Client on your Machine
    • Step 6: Opening the “Terminal Emulator” on the Linux Desktop
    • Step 7: Start your GUI application
      • multi-CPU parallel on a single node specify the number of cores with -t
  • Fluent using Multiple Nodes
    • pre-Step 7: Preparing Fluent Node Connection List
    • Mulit-node Step 7: Starting the Fluent Application

Cloud Resources

• Amazon Web Service (AWS)
  • Centralized Billing and Cost savings
• Google Cloud Platform (GCP)

High Performance Computing (HPC)

• Systems
  • Wendian
    • Wendian Compute Configuration
      • Compute Node Details
      • Filesystem
    • Cooling
  • Mio
    • Students
    • Faculty
    • Mio Compute Configuration
• The Module System
  • Notes on Lmod
  • Common module commands
  • Modules for Compilers
    • MPI Compiler Wrappers
      • Build with GCC and OpenMPI
      • Build with Intel compilers and Openmpi
      • Build with Intel and Intel MPI
  • More on Compilers
    • Intel compilers
      • -ax Suboptions
      • -march Suboptions
        • Build on Mio to run anywhere
        • Build on Mio to run anywhere but include skylake instructions
        • Build on Mio to run anywhere but include skylake instructions, same as above
        • Build on Mio but can only run on skylake nodes
        • Trying to run a skylake code on Mio returns an error
    • GCC Compilers
  • Current Modules
    • Wendian
    • Mio
• Getting Started (New User Guide)
  • Introduction
    • Mines Gitlab Repository for Scripts
  • Connecting to an HPC System via SSH
    • Linux and MacOS
    • Windows
  • Introduction to Command Line and Filesystem
    • Filesystem Basics
    • Command Line Basics for Navigating the Filesystem
    • Choosing a text editor
  • Running Jobs
    • Slurm: HPC Job Scheduler
    • Slurm Job Management and Policy
    • Running your first compiled job - Hello World!
  • An Introduction to the Module System
    • Important Module commands
  • Running jobs with modules
    • Hello World! Using MPI
    • [OPTIONAL] Using Python
      • Creating your own environment using Anaconda
      • Deactivating your conda environment
      • Running Python with Slurm
  • Post-processing and Visualizing Data on HPC
  • Transferring data off HPC systems
    • Transferring files on Windows or without command line
• Connecting to Mines@HPC Systems
  • Setting up SSH keys for easier login
  • Connecting to HPC using Open OnDemand
• Using Anaconda for python environments
  • Setting up base environment
    • Loading the Python module
      • Wendian and Mio
    • Creating and activating a conda environment under your home directory
    • Creating conda environment in a custom location
  • Appliciation-specific conda environment
    • NumPy + SciPy + Matplotlib
    • Tensorflow (GPU)
    • Tensorflow (CPU-only)
  • Cleaning up conda packages
    • Remove all cached packages
  • Proper mpi4py setup to use system MPI on Wendian/Mio
• Running MATLAB on HPC Systems
  • Running an interactive job
  • Submitting a Matlab Job to {Mio, Wendian}
• Advanced Slurm Guide
  • Basics
    • General Slurm Commands
    • sbatch
    • HPC@Mines Specific Commands
    • Wendian-specific Commands
      • Using Charge Accounts
    • Mio-specific commands
  • Checking on jobs using sacct
  • Configuring compute resources
    • Determining node configuration
  • Dependencies
  • Job Arrays
• Knowing your job efficiency
  • XDMoD.mines.edu
    • Using the Efficiency Tab
  • SLURM command ‘seff’
  • Selecting Job ID in XDMoD.mines.edu
    • Example of Rscript using 3-nodes
    • Example of Ansys Fluent (interactively running)
• Parallel Scaling Guide
  • Software Prerequisities for Guide
  • Introduction
  • Prerequisities to use this Guide
  • What is parallel scalability?
  • Before considering parallel scalability
    • Parallelization Support
    • Serial Performance
  • Measuring Parallel Scalability
    • Notation
    • Performance Metrics
      • Wall Time
      • Speed-up
      • Efficiency
      • Load balancing
      • Other metrics
    • Basic Theory
      • Amdahl’s Law
      • Strong Scaling
      • Weak Scaling & Gustafson’s Law
  • Example: Parallel Matrix-Vector Multiplication using Python
    • Serial Matrix-Vector Multiplicaton using Python
      • Pure Python Implementation
      • NumPy Array Implementation
      • NumPy’s built-in Matrix-vector multiplication function: matmul()
      • Using Numba with NumPy
      • Row-wise Parallelization of Matrix-vector multiplication in Python using MPI
    • Slurm Setup
• References
• Archived Guides
  • Programming Guides
  • Scripting Guides

Budget Guidance

• Research Computing Guidance
• Case Study: Researcher using Open Source Finite Element Code FEniCS for modeling reacting flows

Indices and tables

• Index

• Module Index

• Search Page