Merge pull request #142 from ec-jrc/development

Replacement of cython with numba (routing)

.coveragerc

@@ -1,4 +1,3 @@
 [run]
 omit =
-    src/lisflood/hydrological_modules/compile_kinematic_wave_parallel_tools.py
     tests/*

.github/workflows/ci_env.yml

@@ -0,0 +1,47 @@
+name: Lisflood OS Unit Tests
+
+on: [push]
+
+jobs:
+  tests:
+    runs-on: ubuntu-20.04
+    strategy:
+      max-parallel: 5
+
+    steps:
+    - uses: actions/checkout@v3
+    - uses: conda-incubator/setup-miniconda@v2
+      with:
+        auto-update-conda: true
+        python-version: 3.7
+    - name: Conda info
+      shell: bash -el {0}
+      run: conda info
+    - name: Install python and gcc
+      shell: bash -el {0}
+      run: |
+        conda install -c conda-forge python=3.7
+        conda install -c conda-forge gcc=12.1.0
+    - name: Install gdal and pcraster
+      shell: bash -el {0}
+      run: |
+        conda install -c conda-forge gdal pcraster
+    - name: Install dependencies
+      shell: bash -el {0}
+      run: |
+        pip install -r requirements.txt
+    - name: Install lisflood-module
+      shell: bash -el {0}
+      run: |
+        pip install .
+    - name: Check installation
+      shell: bash -el {0}
+      run: |
+        gdal-config --version
+        python -c "from osgeo import gdal; print(gdal.__version__)"
+        conda list
+    - name: Test with pytest
+      shell: bash -el {0}
+      run: |
+        pip install pytest pytest-cov
+        pytest

.gitignore

@@ -17,13 +17,11 @@ npm-debug.log*
 __pycache__/
 build/
 *.so
-kinematic_wave_parallel_tools.html
 lisflood_model.egg-info
 /dist/
 .eggs
 .tox
 /src/dist/
-/src/lisflood/hydrological_modules/kinematic_wave_parallel_tools.c
 .coverage
 *.tox.ini
 .vscode/

Dockerfile

@@ -27,9 +27,6 @@ COPY src/lisfloodSettings_reference.xml /
 COPY LICENSE /
 COPY VERSION /
 
-# Compile kwpt
-RUN cd /lisflood/hydrological_modules && conda run -n lisflood python compile_kinematic_wave_parallel_tools.py build_ext --inplace
-
 # RUN Tests
 COPY tests/. /tests/
 COPY pytest.ini /tests

MANIFEST.in

@@ -1,4 +1,3 @@
-include src/lisflood/hydrological_modules/kinematic_wave_parallel_tools.pyx
 include requirements.txt
 include *.xml
 include VERSION

README.md

@@ -26,7 +26,7 @@ You can use conda environment to easily install dependencies.
 ```bash
 conda create --name lisflood python=3.7 -c conda-forge
 conda activate lisflood
-conda install -c conda-forge pcraster
+conda install -c conda-forge pcraster gdal
 ```
 
 * Install lisflood-model pypi package
@@ -61,26 +61,7 @@ pip install -r requirements.txt
 If you don't use conda but a plain virtualenv, you need to install PCRaster and GDAL by your own and include its python interface in PYTHONPATH environment variable.
 For details, please follow instruction on [official docs](http://pcraster.geo.uu.nl).
 
-
-3. Compile the cython module kinematic_wave_parallel_tool
-
-To compile this Cython module to enable OpenMP multithreading (parallel kinematic wave):
-
-* Delete the files *.so (if any) in directory hydrological-modules  
-
-* Inside the hydrological_modules folder, execute "python compile_kinematic_wave_parallel_tools.py build_ext --inplace"  
-
-Important: the module has to be compiled on the machine where the model is run - the resulting binary is not portable.  
-
-Then in the settings file the option "numberParallelThreadsKinematicWave" may take the following values:
-    - "0"           : auto-detection of the machine/node's number of CPUs (all CPUs are used minus 1) (do not set it if other simulations are running on the same machine/node)
-    - "1"           : serial execution (not parallel)
-    - "2", "3", ... : manual setting of the number of parallel threads.
-                      (if exceeding the number of CPUs, the option is set to "0") -->  
-```xml
-<textvar name="numCPUs_parallelKinematicWave" value="3"/>
-```
-4. Run a cold run for the test catchment
+3. Run a cold run for the test catchment
 
 Now your environment should be set up to run lisflood. Try with a prepared settings file for one of the two test catchments:
 

VERSION

@@ -1 +1 @@
-4.2.0
+4.3.0

docs/3_step2_installation/index.md

@@ -95,32 +95,12 @@ If you don't use conda but a plain virtualenv, you need to install PCRaster and
 For details, please follow instruction on [official docs](https://pcraster.geo.uu.nl/pcraster/4.3.1/documentation/pcraster_project/install.html).
     
 
-3. **Compile the cython module kinematic_wave_parallel_tool**
-   
-   To compile this Cython module to enable OpenMP multithreading (parallel kinematic wave):
-    
-     * Delete the files *.so (if any) in directory hydrological-modules  
-     * Inside the hydrological_modules folder, execute `python compile_kinematic_wave_parallel_tools.py build_ext --inplace`  
-
-   Important: the module has to be compiled on the machine where the model is run - the resulting binary is not portable.  
-  
-   Then in the settings file the option "numberParallelThreadsKinematicWave" may take the following values:
-  
-      * "0"           : auto-detection of the machine/node's number of CPUs (all CPUs are used minus 1) (do not set it if other simulations are running on the same machine/node)
-      * "1"           : serial execution (not parallel)
-      * "2", "3", ... : manual setting of the number of parallel threads.
-                        (if exceeding the number of CPUs, the option is set to "0")
-
-   ```xml
-   <textvar name="numCPUs_parallelKinematicWave" value="3"/>
-   ```
-
-4. **Run a cold run for the test catchment**
+3. **Run a cold run for the test catchment**
 
     Now that your environment should be set up to run lisflood, you may try with a prepared settings file for test catchment included into the tests/data folder:
     
     ```bash
-    python src/lisf1.py tests/data/<TestCatchmentFolder>/settings/cold_day_base.xml
+    python src/lisf1.py tests/data/<TestCatchmentFolder>/settings/cold.xml
     ```
 4. **Run LISFLOOD unit tests**