hpc_python.md

HPC Python

A collection of modules and tools for developing HPC Python scripts. These resources enable python to utilize a GPU or multi-node execution.

Resources

Concurrency/Parallelism

• Dask
  • Dask-MPI: deploy workers across MPI ranks
• MPI4Py: Python MPI bindings
• Ray: Distributed Python

GPU

• CuPy: GPU accelerated NumPy implementation
• cuDF: GPU accelerated dataframes, efficient for use with strings and numerics
• CuML: Scikit-learn and other ML on GPU

Speed

• UCX-Py: UCX bindings to take advantage of NVLINK or Infiniband
• Numba: Numba makes Python code fast
• Modin Drop in replacement for Pandas and Dask, can distribute operations

Integrate

• pvpython
  • pvpython examples: ParaView/Python Recipes