Updated documentation for installation, development setup etc.

CHANGELOG.md

@@ -8,10 +8,10 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 - Added Poetry for dependency and build management
 - Switched from `setup.py` and `setup.cfg` to `pyproject.toml` for centralized metadata, added relevant tests
 - Updated GitHub Action workflows to use Poetry
-- Updated dependencies, most notably NumPy is no longer version-locked as scikit-opt is no longer a dependency [TODO]: update instructions for installation
+- Updated dependencies, most notably NumPy is no longer version-locked as scikit-opt is no longer a dependency
 - Documentation now uses `pyproject.toml` metadata, minor fixes and changes to be compatible with updated dependencies
 - Set up Nox for testing on all supported Python versions in isolated environments
-- Added linting information, VS Code settings and recommendations [TODO]: update documentation (contributing etc.)
+- Added linting information, VS Code settings and recommendations
 - Dropped Python 3.7 support
 
 ## [0.4.5] - 2023-06-01

CONTRIBUTING.rst

@@ -9,20 +9,18 @@ Not all contributions are code, creating an issue also helps us to improve. When
 * Describe what you expected to happen.
 * If possible, include a minimal example to help us reproduce the issue.
 * Describe what actually happened, including the output of any errors printed.
-* List the version of Python, CUDA or OpenCL, and C compiler, if applicable. 
+* List the version of Python, CUDA or OpenCL, and C compiler, if applicable.
 
 Contributing Code
 -----------------
 For contributing code to Kernel Tuner please select an issue to work on or create a new issue to propose a change or addition. For significant changes, it is required to first create an issue and discuss the proposed changes. Then fork the repository, create a branch, one per change or addition, and create a pull request.
 
-Kernel Tuner follows the Google Python style guide, with Sphinxdoc docstrings for module public functions. Please use `pylint` to check your Python changes.
+Kernel Tuner follows the Google Python style guide, with Sphinxdoc docstrings for module public functions.
 
 Before creating a pull request please ensure the following:
 
-* You have written unit tests to test your additions and all unit tests pass
+* You have written unit tests to test your additions and all unit tests pass (run `nox`)
 * The examples still work and produce the same (or better) results
-* The code is compatible with Python 3.5 or newer
-* You have run `pylint` to check your code
 * An entry about the change or addition is created in CHANGELOG.md
 * Any matching entries in the roadmap.md are updated/removed
 
@@ -32,20 +30,24 @@ have look at the `design documentation
 
 Development setup
 -----------------
-You can install the packages required to run the tests using:
+Afer cloning, you can install the packages required to run the tests using:
 
 .. code-block:: bash
 
-    pip install -e .[dev]
+    poetry install --no-root --with test,docs
+    pip install -e .
 
 After this command you should be able to run the tests and build the documentation.
 See below on how to do that. The ``-e`` flag installs the package in *development mode*.
 This means files are not copied, but linked to, such that your installation tracks
 changes in the source files.
+Additionally you can install any of the optional runtime dependencies by appending e.g. `--extras cuda,opencl` to the Poetry command.
+If you want to go all-out, use `--all-extras`.
+
 
 Running tests
 -------------
-To run the tests you can use ``pytest -v test/`` in the top-level directory.
+To run the tests you can use ``pytest`` (to run against the local Python version) and ``nox``` (to run against all supported Python versions) in the top-level directory.
 
 Note that tests that require PyCuda and/or a CUDA capable GPU will be skipped if these
 are not installed/present. The same holds for tests that require PyOpenCL, Cupy, Nvidia CUDA.
@@ -61,14 +63,13 @@ Building documentation
 Documentation is located in the ``doc/`` directory. This is where you can type
 ``make html`` to generate the html pages in the ``doc/build/html`` directory.
 The source files used for building the documentation are located in
-``doc/source``. 
+``doc/source``.
 To locally inspect the documentation before committing you can browse through
 the documentation pages generated locally in ``doc/build/html``.
 
-To make sure you have all the dependencies required to build the documentation,
-you can install the extras using ``pip install -e .[doc]``. Pandoc is also required,
-you can install pandoc on ubuntu using ``sudo apt install pandoc``, for different
-setups please see `pandoc's install documentation <https://pandoc.org/installing.html>`__.
+To make sure you have all the dependencies required to build the documentation, at least those in ``--with docs``.
+Pandoc is also required, you can install pandoc on Ubuntu using ``sudo apt install pandoc`` and on Mac using ``brew install pandoc``.
+For different setups please see `pandoc's install documentation <https://pandoc.org/installing.html>`__.
 
 The documentation pages hosted online are built automatically using GitHub actions.
 The documentation pages corresponding to the master branch are hosted in /latest/.

INSTALL.rst

@@ -1,15 +1,15 @@
 Installation
 ============
 
-The Kernel Tuner requires several packages to be installed. First of all, you need a 
-working Python version, several Python packages, and optionally CUDA and/or OpenCL 
+The Kernel Tuner requires several packages to be installed. First of all, you need a
+working Python version, several Python packages, and optionally CUDA and/or OpenCL
 installations. All of this is explained in detail in this guide.
 
 
 Python
 ------
 
-You need a Python installation. I recommend using Python 3 and 
+You need a Python installation. I recommend using Python 3 and
 installing it with `Miniconda <https://conda.io/miniconda.html>`__.
 
 Linux users could type the following to download and install Python 3 using Miniconda:
@@ -25,11 +25,11 @@ is developed to be fully compatible with Python 3.6 and newer.
 Installing Python Packages
 --------------------------
 
-Note that when you are using a native Python installation, the `pip` command used 
-Kernel Tuner and its dependencies require `sudo` rights for system wide installation. 
+Note that when you are using a native Python installation, the `pip` command used
+Kernel Tuner and its dependencies require `sudo` rights for system wide installation.
 
 Sudo rights are typically not required when using Miniconda or virtual environments.
-You could also use e.g. the `--user` or `--prefix` option of `pip` to install into 
+You could also use e.g. the `--user` or `--prefix` option of `pip` to install into
 your home directory,
 this requires that your home directory is on your `$PYTHONPATH` environment variable
 (see for further details the pip documentation).
@@ -45,15 +45,15 @@ There are also optional dependencies, explained below.
 CUDA and PyCUDA
 ---------------
 
-Installing CUDA and PyCUDA is optional, because you may want to only use Kernel 
-Tuner for tuning OpenCL or C kernels. 
+Installing CUDA and PyCUDA is optional, because you may want to only use Kernel
+Tuner for tuning OpenCL or C kernels.
 
-If you want to use the Kernel Tuner to tune 
-CUDA kernels you will first need to install the CUDA toolkit 
-(https://developer.nvidia.com/cuda-toolkit). A recent version of the 
-CUDA toolkit (and the PyCUDA Python bindings for CUDA) are 
-recommended (older version may work, but may not support all features of 
-Kernel Tuner). 
+If you want to use the Kernel Tuner to tune
+CUDA kernels you will first need to install the CUDA toolkit
+(https://developer.nvidia.com/cuda-toolkit). A recent version of the
+CUDA toolkit (and the PyCUDA Python bindings for CUDA) are
+recommended (older version may work, but may not support all features of
+Kernel Tuner).
 
 It's very important that you install the CUDA toolkit before trying to install PyCuda.
 
@@ -72,7 +72,7 @@ Or you could install Kernel Tuner and PyCUDA together if you haven't done so alr
 If you run into trouble with installing PyCuda, make sure you have CUDA installed first.
 Also make sure that the Python package Numpy is already installed, e.g. using `pip install numpy`.
 
-If you retry the ``pip install pycuda`` command, you may need to use the 
+If you retry the ``pip install pycuda`` command, you may need to use the
 ``--no-cache-dir`` option to ensure the pycuda installation really starts over and not continues
 from an installation that is failing.
 
@@ -82,8 +82,8 @@ If this fails, I recommend to see the PyCuda installation guide (https://wiki.ti
 OpenCL and PyOpenCL
 -------------------
 
-Before we can install PyOpenCL you'll need an OpenCL compiler. There are several 
-OpenCL compilers available depending on the OpenCL platform you want to your 
+Before we can install PyOpenCL you'll need an OpenCL compiler. There are several
+OpenCL compilers available depending on the OpenCL platform you want to your
 code to run on.
 
 * `AMD APP SDK <https://rocmdocs.amd.com/en/latest/Programming_Guides/Opencl-programming-guide.html>`__
@@ -94,7 +94,7 @@ code to run on.
 
 You can also look at this `OpenCL Installation Guide <https://wiki.tiker.net/OpenCLHowTo>`__ for PyOpenCL.
 
-As with the CUDA toolkit, recent versions of one or more of the above OpenCL SDK's and 
+As with the CUDA toolkit, recent versions of one or more of the above OpenCL SDK's and
 PyOpenCL are recommended to support all features of the Kernel Tuner.
 
 After you've installed your OpenCL compiler of choice you can install PyOpenCL using:
@@ -114,7 +114,7 @@ If this fails, please see the PyOpenCL installation guide (https://wiki.tiker.ne
 HIP and PyHIP
 -------------
 
-Before we can install PyHIP, you'll need to have the HIP runtime and compiler installed on your system. 
+Before we can install PyHIP, you'll need to have the HIP runtime and compiler installed on your system.
 The HIP compiler is included as part of the ROCm software stack. Here is AMD's installation guide:
 
 * `ROCm Documentation: HIP Installation Guide <https://docs.amd.com/bundle/HIP-Installation-Guide-v5.3/page/Introduction_to_HIP_Installation_Guide.html>`__
@@ -134,36 +134,47 @@ Alternatively, you can install PyHIP from the source code. First, clone the repo
 Then, navigate to the repository directory and run the following command to install:
 
 .. code-block:: bash
-    
+
     python setup.py install
 
 Installing the git version
 --------------------------
 
-You can also install from the git repository. This way you also get the 
+You can also install from the git repository. This way you also get the
 examples.
 
 .. code-block:: bash
 
     git clone https://github.com/benvanwerkhoven/kernel_tuner.git
     cd kernel_tuner
-    pip install .
+    poetry install
 
-You can install Kernel Tuner with several optional dependencies, the full list is:
+You can install Kernel Tuner with several optional dependencies.
+In this we differentiate between development and runtime dependencies.
+The development dependencies are ``test`` and ``docs``, and can be installed by appending e.g. ``--with test,docs``.
+The runtime dependencies are:
 
 - `cuda`: install pycuda along with kernel_tuner
 - `opencl`: install pycuda along with kernel_tuner
 - `hip`: install pyhip along with kernel_tuner
-- `doc`: installs packages required to build the documentation
 - `tutorial`: install packages required to run the guides
-- `dev`: install everything you need to start development on Kernel Tuner
+
+These can be installed by appending e.g. ``--extras cuda,opencl``.
+If you want to go all-out, use ``--all-extras``.
 
 For example, use:
 ```
-pip install .[dev,cuda,opencl]
+pip install --with test,docs --extras cuda,opencl
 ```
 To install Kernel Tuner along with all the packages required for development.
 
+You can also install Kernel Tuner in development mode by installing the packages and using a local, editable pip install:
+
+.. code-block:: bash
+
+    poetry install --no-root --with test,docs
+    pip install -e .
+
 
 Dependencies for the guides
 ---------------------------

README.rst

@@ -3,15 +3,16 @@ Kernel Tuner
 
 |Build Status| |CodeCov Badge| |PyPi Badge| |Zenodo Badge| |SonarCloud Badge| |OpenSSF Badge| |FairSoftware Badge|
 
-Kernel Tuner simplifies the software development of optimized and auto-tuned GPU programs, by enabling Python-based unit testing of GPU code and making it easy to develop scripts for auto-tuning GPU kernels. This also means no extensive changes and no new dependencies are required in the kernel code. The kernels can still be compiled and used as normal from any host programming language.
+Kernel Tuner simplifies the software development of optimized and auto-tuned GPU programs, by enabling Python-based unit testing of GPU code and making it easy to develop scripts for auto-tuning GPU kernels.
+This also means no extensive changes and no new dependencies are required in the kernel code.
+The kernels can still be compiled and used as normal from any host programming language.
 
 Kernel Tuner provides a comprehensive solution for auto-tuning GPU programs, supporting auto-tuning of user-defined parameters in both host and device code, supporting output verification of all benchmarked kernels during tuning, as well as many optimization strategies to speed up the tuning process.
 
 Documentation
 -------------
 
-The full documentation is available
-`here <https://kerneltuner.github.io/kernel_tuner/stable/index.html>`__.
+The full documentation is available `here <https://kerneltuner.github.io/kernel_tuner/stable/index.html>`__.
 
 Installation
 ------------
@@ -38,7 +39,7 @@ Or all:
 - ``pip install kernel_tuner[cuda,opencl,hip]``
 
 More information about how to install Kernel Tuner and its
-dependencies can be found in the `installation guide 
+dependencies can be found in the `installation guide
 <http://kerneltuner.github.io/kernel_tuner/stable/install.html>`__.
 
 Example usage
@@ -83,12 +84,12 @@ The exact same Python code can be used to tune an OpenCL kernel:
     }
     """
 
-The Kernel Tuner will detect the kernel language and select the right compiler and 
-runtime. For every kernel in the parameter space, the Kernel Tuner will insert C 
-preprocessor defines for the tunable parameters, compile, and benchmark the kernel. The 
-timing results will be printed to the console, but are also returned by tune_kernel to 
-allow further analysis. Note that this is just the default behavior, what and how 
-tune_kernel does exactly is controlled through its many `optional arguments 
+The Kernel Tuner will detect the kernel language and select the right compiler and
+runtime. For every kernel in the parameter space, the Kernel Tuner will insert C
+preprocessor defines for the tunable parameters, compile, and benchmark the kernel. The
+timing results will be printed to the console, but are also returned by tune_kernel to
+allow further analysis. Note that this is just the default behavior, what and how
+tune_kernel does exactly is controlled through its many `optional arguments
 <http://kerneltuner.github.io/kernel_tuner/stable/user-api.html#kernel_tuner.tune_kernel>`__.
 
 You can find many - more extensive - example codes, in the
@@ -99,19 +100,19 @@ documentation pages <http://kerneltuner.github.io/kernel_tuner/stable/index.html
 Search strategies for tuning
 ----------------------------
 
-Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently 
-implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC, 
-COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm 
+Kernel Tuner supports many optimization algorithms to accelerate the auto-tuning process. Currently
+implemented search algorithms are: Brute Force (default), Nelder-Mead, Powell, CG, BFGS, L-BFGS-B, TNC,
+COBYLA, SLSQP, Random Search, Basinhopping, Differential Evolution, a Genetic Algorithm, Particle Swarm
 Optimization, the Firefly Algorithm, Simulated Annealing, Dual Annealing, Iterative Local Search,
 Multi-start Local Search, and Bayesian Optimization.
 
 .. image:: doc/gemm-amd-summary.png
     :width: 100%
     :align: center
 
-Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method 
-you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping"``. 
-For a full overview of the supported search strategies and methods please see the 
+Using a search strategy is easy, you only need to specify to ``tune_kernel`` which strategy and method
+you would like to use, for example ``strategy="genetic_algorithm"`` or ``strategy="basinhopping"``.
+For a full overview of the supported search strategies and methods please see the
 Kernel Tuner documentation on `Optimization Strategies <https://kerneltuner.github.io/kernel_tuner/stable/optimization.html>`__.
 
 Tuning host and kernel code
@@ -172,7 +173,7 @@ If you use Kernel Tuner in research or research software, please cite the most r
       year = {2021},
       url = {https://arxiv.org/abs/2111.14991}
     }
-    
+
     @article{schoonhoven2022benchmarking,
       title={Benchmarking optimization algorithms for auto-tuning GPU kernels},
       author={Schoonhoven, Richard and van Werkhoven, Ben and Batenburg, K Joost},
@@ -196,7 +197,7 @@ If you use Kernel Tuner in research or research software, please cite the most r
    :target: https://github.com/KernelTuner/kernel_tuner/actions/workflows/python-app.yml
 .. |CodeCov Badge| image:: https://codecov.io/gh/KernelTuner/kernel_tuner/branch/master/graph/badge.svg
    :target: https://codecov.io/gh/KernelTuner/kernel_tuner
-.. |PyPi Badge| image:: https://img.shields.io/pypi/v/kernel_tuner.svg?colorB=blue 
+.. |PyPi Badge| image:: https://img.shields.io/pypi/v/kernel_tuner.svg?colorB=blue
    :target: https://pypi.python.org/pypi/kernel_tuner/
 .. |Zenodo Badge| image:: https://zenodo.org/badge/54894320.svg
    :target: https://zenodo.org/badge/latestdoi/54894320