Skip to content

Latest commit

 

History

History
63 lines (48 loc) · 3.18 KB

README.md

File metadata and controls

63 lines (48 loc) · 3.18 KB
Raw

This directory contains tutorials and examples of Cabana (https://github.com/ECP-copa/Cabana) for Fortran.

Tutorial:

Explanations of using the basic data structures and parallel capabilities of Kokkos and Cabana with Fortran

  • Hello World
  • Tuples
  • Struct-of-Arrays
  • AoSoA
  • Parallel for
  • Kokkos::parallel_for

Advanced examples:

Kernels that can be readily adopted for users wanting to build a Fortran PIC code or incorporate Cabana kernels into an existing Fortran PIC code.

  • parallel_for
  • Push
  • Scatter
  • Unmanaged aosoa

cabana_cpp_interface.cpp

This file contains all of the Kokkos/Cabana code, which is in C++. It contains an macro enabling usage of the Kokkos parallel_for both generally and specifically to a particle AoSoA. Users must specify the particle properties here.

cabana_fortran_macros.h

This include file contains macros used in the Fortran code:

  • ATTR_DEVICE: Adds "attributes(device)" in the GPU version to specify execution space. Other Cuda options available: ATTR_HOST, ATTR_GLOBAL, ATTR_SHARED, ATTR_CONSTANT, ATTR_PINNED
  • COPY_TO_DEVICE(err,icount,d_array,h_array): Copies from host to device if GPU version, otherwise simple copy
  • COPY_TO_HOST(err,icount,h_array,d_array): Copies from device to host if GPU version, otherwise simple copy
  • CABANA_COUNT_KIND: Variable type needed for icount in the above macros
  • FCABANA_DEVICE: Specifies that a variable or array is on the device if GPU version
  • CABANA_DEF_THREAD_NUM: Declares OMP_GET_THREAD_NUM in OpenMP version
  • CABANA_GET_THREAD_NUM(x): Gets OMP thread number
  • CABANA_ADD(x,y): Normal addition if on CPU; AtomicAdd if on GPU.
  • CABANA_DEF_MAX_THREADS: Declates OMP_GET_MAX_THREADS in OpenMP version
  • CABANA_GET_MAX_THREADS(x): Gets OMP number of threads
  • PARTICLE_OP_INTERFACE(C_FUNC): Interfaces to the corresponding C++ routine in cabana_cpp_interface.cpp or particle_ops.h
  • MISC_OP_INTERFACE(C_FUNC): Interfaces to the corresponding C++ routine in misc_ops.h

cabana_fortran_wrappers.F90

Contains Fortran wrappers for Cabana/Kokkos C++ routines:

  • cabana_initialize: Cabana::initialize
  • cabana_finalize: Cabana::finalize
  • kokkos_fence: Kokkos::fence

ptl_module.F90

Users must specify the particle properties here and they must correspond to those in cabana_cpp_interface.cpp. The particle initialization and some utilities are also here.

particle_ops.h

This is a list of particle operations. In these operations, the fortran kernel receives the particle data, number of structures in the SoA, and the parallel_for index as arguments. Users can add their own operations to the list and easily create custom kernels by analogy with existing ones.

grid_module.F90

An example grid module that a user might have.

update_phase_module.F90

The user-specified physics of the push.

push_module.F90

The push kernel. Currently does nothing interesting besides "subcycling" - performing the push multiple times per time step. Could be expanded to include multiple algorithms to choose from.

scatter_module.F90

The scatter kernel. Currently decides which cell to write to arbitrarily, and deposits a "1" in the specified cell.