Skip to content

Commit

Permalink
Add configuration for NCSU's Hazel cluster
Browse files Browse the repository at this point in the history
With thanks to @jtande!
  • Loading branch information
khk-globus committed Nov 14, 2024
1 parent 55e0759 commit 9ce329f
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 1 deletion.
27 changes: 27 additions & 0 deletions docs/endpoints/configs/ncsu-hazel.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
display_name: Hazel HTEX
engine:
type: GlobusComputeEngine

address:
type: address_by_interface
ifname: ens5f0

provider:
type: LSFProvider
launcher:
type: SingleNodeLauncher

walltime: 00:30:00
nodes_per_block: 1
init_blocks: 1
max_blocks: 1

queue: standard

worker_init: |
module load conda
conda activate /path/to/your/condaenv/conda_env
scheduler_options: "# BSUB -R span[hosts=1]"
cores_per_node: 12
cmd_timeout: 60
20 changes: 19 additions & 1 deletion docs/endpoints/endpoint_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,17 @@ scheduler, and uses the ``SrunLauncher`` to launch workers.
.. literalinclude:: configs/faster.yaml
:language: yaml


Hazel (NCSU)
^^^^^^^^^^^^

The Hazel cluster uses the LSF scheduler; the following configuration assumes the user
is running on a login node, uses the |LSFProvider|_ and the |SingleNodeLauncher|_.

.. literalinclude:: configs/ncsu-hazel.yaml
:language: yaml


Pinning Workers to devices
^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -233,5 +244,12 @@ specific identity assigned: ``CUDA_VISIBLE_DEVICES``, ``ROCR_VISIBLE_DEVICES``,
.. |HighThroughputExecutor| replace:: ``HighThroughputExecutor``
.. _HighThroughputExecutor: https://parsl.readthedocs.io/en/stable/stubs/parsl.executors.HighThroughputExecutor.html

.. |Providers| replace:: ``Providers``
.. _Providers: https://parsl.readthedocs.io/en/stable/reference.html#providers
.. |LSFProvider| replace:: ``LSFProvider``
.. _LSFProvider: https://parsl.readthedocs.io/en/latest/stubs/parsl.providers.LSFProvider.html
.. |SingleNodeLauncher| replace:: ``SingleNodeLauncher``
.. _SingleNodeLauncher: https://parsl.readthedocs.io/en/latest/stubs/parsl.launchers.SingleNodeLauncher.html

.. |GlobusMPIEngine| replace:: ``GlobusMPIEngine``
.. _GlobusMPIEngine: reference/mpi_engine.html
.. _GlobusMPIEngine: reference/mpi_engine.html

0 comments on commit 9ce329f

Please sign in to comment.