diff --git a/README.md b/README.md index e3078ffd..8f5d3682 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ Other useful resources ## Quick start You can download code and datasets for testing the model. -Follow this instruction for a basic test (Drina catchment, included in this repository under [tests/data/Drina](https://github.com/ec-jrc/lisflood-code/tree/master/tests/data/Drina)) +Follow this instruction for a basic test (included in this repository under [tests/data/TestCatchment1](https://github.com/ec-jrc/lisflood-code/tree/master/tests/data/TestCatchment1)) 1. Clone the master branch of this repository (you need to have git installed on your machine). @@ -26,8 +26,9 @@ Follow this instruction for a basic test (Drina catchment, included in this repo git clone --single-branch --branch master https://github.com/ec-jrc/lisflood-code.git ``` -2. Install requirements into a python virtualenv. -We recommend to follow the instructions on [virtualenv docs](https://virtualenv.pypa.io/en/latest/). Assuming you activated your virtual environment: +2. Install requirements into a Python 3 virtualenv. +We recommend to follow the instructions on [virtualenv docs](https://virtualenv.pypa.io/en/latest/). +Assuming you've activated your virtual environment, you can now install requirements with pip: ```bash cd lisflood-code # move into lisflood-code project directory @@ -48,7 +49,7 @@ To compile this Cython module to enable OpenMP multithreading (parallel kinemati * Delete the files *.so (if any) in directory hydrological-modules -* Inside the hydrological_modules folder, execute "python2 compile_kinematic_wave_parallel_tools.py build_ext --inplace" +* Inside the hydrological_modules folder, execute "python compile_kinematic_wave_parallel_tools.py build_ext --inplace" Important: the module has to be compiled on the machine where the model is run - the resulting binary is not portable. @@ -60,15 +61,15 @@ Then in the settings file the option "numberParallelThreadsKinematicWave" may ta ```xml ``` -4. Run a cold run for the Drina test catchment +4. Run a cold run for the test catchment -Now your environment should be set up to run lisflood. Try with a prepared settings file for Drina catchment: +Now your environment should be set up to run lisflood. Try with a prepared settings file for one of the two test catchments: ```bash -python src/lisf1.py tests/data/Drina/settings/lisfloodSettings_cold_day_base.xml +python src/lisf1.py tests/data/TestCatchment1/settings/lisfloodSettings_cold_day_base.xml ``` -If the command above successed without errors, producing dis.nc into tests/data/Drina/outputs folder, your lisflood installation was correct. +If the command above successed without errors, producing dis.nc into tests/data/TestCatchment1/outputs folder, your lisflood installation was correct. ### Docker image @@ -80,26 +81,26 @@ First, you pull image from repository. docker pull efas/lisflood:latest ``` -Copy Drina catchment files from container to your host, using mapped directories. +Copy catchment files from container to your host, using mapped directories. ```bash docker run -v /absolute_path/to/my/local/folder:/usecases efas/lisflood:latest usecases ``` -After this command, you can find all files to run a test against Drina catchment under the directory you mapped: `/absolute_path/to/my/local/folder/Drina` +After this command, you can find all files to run a test against a catchment under the directory you mapped: `/absolute_path/to/my/local/folder/TestCatchment1` -Now, you can run LISFLOOD as a docker container to test the Drina catchment. Only thing you need to do is to map the Drina folder to the container folder `input`, by using -v option. +Now, you can run LISFLOOD as a docker container to test included catchments. Only thing you need to do is to map the TestCatchment1 folder to the container folder `input`, by using -v option. In the XML settings file, all paths are adjusted to be relative to the very same settings file, so you don't need to edit paths, as long as you keep same folders structure. Execute the following to run the simulation: ```bash -docker run -v /absolute_path/to/my/local/folder/Drina:/input efas/lisflood /input/settings/lisfloodSettings_cold_day_base.xml +docker run -v /absolute_path/to/my/local/folder/TestCatchment1:/input efas/lisflood /input/settings/cold_day_base.xml ``` -Once LISFLOOD finished, you can find reported maps in `/absolute_path/to/my/local/folder/Drina/outputs/` folder. +Once LISFLOOD finished, you can find reported maps in `/absolute_path/to/my/local/folder/TestCatchment1/outputs/` folder. ### Pypi packaged LISFLOOD @@ -112,13 +113,13 @@ pip install lisflood-model Command above will also install the executable `lisflood` in the virtualenv, so that you can run LISFLOOD with the following: ```bash -lisflood /absolute_path/to/my/local/folder/Drina/settings/lisfloodSettings_cold_day_base.xml +lisflood /absolute_path/to/my/local/folder/TestCatchment1/settings/lisfloodSettings_cold_day_base.xml ``` ## Collaborate If you find an issue in our code, plese follow the [GitHub flow](https://guides.github.com/introduction/flow/) to propose your changes (Fork, commit your changes and ask for a Pull Request). -When you develop, you need to run our "acceptance" tests. We have a couple of tests (domains are Drina and Madeira catchments) that can run with tox on py27, py36, py37 environments. +When you develop, you need to run our "acceptance" tests. We have two test catchments, that can run with tox on py27, py36, py37 environments. Simply execute `tox` on comman line from project folder. Tox tests can last minutes. You can also just use pytest and run tests in a single environment (e.g. Python 3.7). diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh index 69fd6001..d05a63a1 100755 --- a/docker-entrypoint.sh +++ b/docker-entrypoint.sh @@ -4,8 +4,8 @@ set -e if [[ "$1" = 'usecases' ]]; then mkdir -p /usecases echo "Copying maps to /input/Drina..." - cp -R /tests/data/Drina /usecases/ - chmod a+w /usecases/Drina/settings/*.xml + cp -R /tests/data/TestCatchment1 /usecases/ + chmod a+w /usecases/TestCatchment1/settings/*.xml else exec python /lisf1.py "$@" fi diff --git a/requirements.txt b/requirements.txt index 24a2aff7..02aabc41 100644 --- a/requirements.txt +++ b/requirements.txt @@ -10,3 +10,4 @@ xarray pyproj nine future +lisflood-utilities diff --git a/src/lisf1.py b/src/lisf1.py index c66bd37c..3ceff19a 100644 --- a/src/lisf1.py +++ b/src/lisf1.py @@ -26,7 +26,6 @@ ###################################################################### """ -# to work with the new grid engine JRC - workaround with error on pyexpat import sys from lisflood.main import main diff --git a/src/lisflood/Lisflood_dynamic.py b/src/lisflood/Lisflood_dynamic.py index 3ff3a384..ef884fa4 100644 --- a/src/lisflood/Lisflood_dynamic.py +++ b/src/lisflood/Lisflood_dynamic.py @@ -50,10 +50,8 @@ def dynamic(self): if i == 1: # flag for netcdf output for all, steps and end _ = CDFFlags(uuid.uuid4()) # init CDF flags - # globals.cdfFlag = [0, 0, 0, 0 ,0 ,0,0] self.TimeSinceStart = self.currentTimeStep() - self.firstTimeStep() + 1 - if flags['loud']: print("%-6i %10s" % (self.currentTimeStep(), self.CalendarDate.strftime("%d/%m/%Y %H:%M"))) else: @@ -64,6 +62,10 @@ def dynamic(self): # Print step number and date to console sys.stdout.write("\r%d" % i), sys.stdout.write("%s" % " - "+self.CalendarDate.strftime("%d/%m/%Y %H:%M")) sys.stdout.flush() + if i == self.nrTimeSteps(): + # last timestep. Send a new line to the terminal for polishness + sys.stdout.write('\n') + sys.stdout.flush() # ************************************************************ """ up to here it was fun, now the real stuff starts diff --git a/src/lisflood/OptionTserieMaps.xml b/src/lisflood/OptionTserieMaps.xml deleted file mode 100644 index fd79d2de..00000000 --- a/src/lisflood/OptionTserieMaps.xml +++ /dev/null @@ -1,916 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/src/lisflood/__init__.py b/src/lisflood/__init__.py index 0de3436a..905b8933 100644 --- a/src/lisflood/__init__.py +++ b/src/lisflood/__init__.py @@ -1,7 +1,7 @@ -version = (2, 9, 6) +version = (2, 10, 0) __authors__ = "Ad de Roo, Emiliano Gelati, Peter Burek, Johan van der Knijff, Niko Wanders" __version__ = '.'.join(list(map(str, version))) -__date__ = "30 August 2019" +__date__ = "03/09/2019" __copyright__ = "Copyright 2019, European Commission - Joint Research Centre" __maintainer__ = "Emiliano Gelati, Domenico Nappo, Valerio Lorini, Lorenzo Mentaschi, Ad de Roo" __status__ = "Operation" diff --git a/src/lisflood/global_modules/checkers.py b/src/lisflood/global_modules/checkers.py new file mode 100644 index 00000000..4df11048 --- /dev/null +++ b/src/lisflood/global_modules/checkers.py @@ -0,0 +1,63 @@ +import importlib +import inspect + +from .errors import LisfloodError +from ..hydrological_modules import HydroModule +from ..hydrological_modules import (surface_routing, evapowater, snow, routing, leafarea, inflow, waterlevel, + waterbalance, wateruse, waterabstraction, lakes, riceirrigation, indicatorcalc, + landusechange, frost, groundwater, miscInitial, soilloop, soil, + reservoir, transmission) + + +class ModulesInputs: + root_package = 'lisflood.hydrological_modules' + # dict representing modules activated per option + lisflood_modules = { + # not optional modules + 'all': [surface_routing, snow, routing, leafarea, landusechange, + frost, groundwater, miscInitial, soil], + # list of modules activated by options in LisFlood settings.xml + 'inflow': [inflow], + 'wateruse': [wateruse], + 'groundwaterSmooth': [waterabstraction], + 'wateruseRegion': [waterabstraction], + 'drainedIrrigation': [soilloop, soil], + 'riceIrrigation': [riceirrigation, waterabstraction], + 'indicator': [lakes, indicatorcalc, waterabstraction], + 'openwaterevapo': [evapowater], + 'varfractionwater': [evapowater], + 'TransientLandUseChange': [landusechange, indicatorcalc, waterabstraction], + 'simulateLakes': [lakes, indicatorcalc, routing, waterabstraction, waterbalance], + 'simulateReservoirs': [reservoir, indicatorcalc, routing, waterabstraction, waterbalance], + 'simulatePF': [soilloop, soil], + 'simulateWaterLevels': [waterlevel], + 'TransLoss': [transmission], + 'gridSizeUserDefined': [miscInitial], + } + + @classmethod + def check(cls, settings): + """ + + :param settings: LisSettings object + """ + binding = settings.binding + res = False + total_checks = 0 + for option, modules in cls.lisflood_modules.items(): + if not (binding.get(option) or option == 'all'): + # module is not activated + continue + for hydro_module in modules: + # hydro_module = importlib.import_module('{}.{}'.format(cls.root_package, m)) + # hydro_module_str = '{}.{}'.format(cls.root_package, m) + clzzs = [obj for n, obj in inspect.getmembers(hydro_module) + if inspect.isclass(obj) and issubclass(obj, HydroModule) + and obj is not HydroModule and str(obj.__module__) == hydro_module.__name__] + total_checks += len(clzzs) + for clz in clzzs: + res += clz.check_input_files(option) + if res < total_checks: + # some checks failed + raise LisfloodError('\n\nMissing files to run LisFlood according activated modules. ' + 'Please check your settings file {}'.format(settings.settings_path)) diff --git a/src/lisflood/hydrological_modules/__init__.py b/src/lisflood/hydrological_modules/__init__.py index e69de29b..ed5e73e2 100644 --- a/src/lisflood/hydrological_modules/__init__.py +++ b/src/lisflood/hydrological_modules/__init__.py @@ -0,0 +1,71 @@ +""" + +Copyright 2019 European Union + +Licensed under the EUPL, Version 1.2 or as soon they will be approved by the European Commission +subsequent versions of the EUPL (the "Licence"); + +You may not use this work except in compliance with the Licence. +You may obtain a copy of the Licence at: + +https://joinup.ec.europa.eu/sites/default/files/inline-files/EUPL%20v1_2%20EN(1).txt + +Unless required by applicable law or agreed to in writing, +software distributed under the Licence is distributed on an "AS IS" basis, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the Licence for the specific language governing permissions and limitations under the Licence. + +""" +from __future__ import absolute_import, print_function, division +from nine import range + +import os + +from ..global_modules.errors import LisfloodWarning +from ..global_modules.settings import LisSettings + + +def is_number(v): + if not v: + return False + try: + float(v) + except ValueError: + return False + else: + return True + + +def is_path(v): + if not v: + return False + path_no_ext, current_ext = os.path.splitext(v) + v_alt = '{}.{}'.format(path_no_ext, 'nc' if current_ext in ('.map', '') else 'map') + a = (os.path.exists(v) and os.path.isfile(v)) or os.access(v, os.W_OK) + b = (os.path.exists(v_alt) and os.path.isfile(v_alt)) or os.access(v_alt, os.W_OK) + return a or b + + +class HydroModule(object): + input_files_keys = None + module_name = None + + @classmethod + def check_input_files(cls, option): + settings = LisSettings.instance() + binding = settings.binding + ok = True + keys = cls.input_files_keys[option] + for k in keys: + msg = None + k_ok = True + if not binding.get(k): + msg = '[{}]: setting "{}" is missing in settings file {}'.format(cls.module_name, k, settings.settings_path) + k_ok = False + elif not (is_path(binding[k]) or is_number(binding[k])): + k_ok = False + msg = '[{}]: setting {} refers to a non existing path or a not well-formed float value: {}'.format(cls.module_name, k, binding[k]) + if not k_ok: + ok = False + print(LisfloodWarning(msg)) + return ok diff --git a/src/lisflood/hydrological_modules/compile_kinematic_wave_tools.md b/src/lisflood/hydrological_modules/compile_kinematic_wave_tools.md index c4c8de18..9ba572b4 100644 --- a/src/lisflood/hydrological_modules/compile_kinematic_wave_tools.md +++ b/src/lisflood/hydrological_modules/compile_kinematic_wave_tools.md @@ -6,7 +6,7 @@ To compile this Cython module to enable OpenMP multithreading (parallel kinemati 1) Delete the files *.so (if any) in directory hydrological-modules -2) Inside the hydrological_modules folder, execute "python2 compile_kinematic_wave_parallel_tools.py build_ext --inplace" +2) Inside the hydrological_modules folder, execute "python compile_kinematic_wave_parallel_tools.py build_ext --inplace" Important: the module has to be compiled on the machine where the model is run - the resulting binary is not portable. diff --git a/src/lisflood/hydrological_modules/evapowater.py b/src/lisflood/hydrological_modules/evapowater.py index 4738d994..fbab3c0d 100644 --- a/src/lisflood/hydrological_modules/evapowater.py +++ b/src/lisflood/hydrological_modules/evapowater.py @@ -22,14 +22,20 @@ from ..global_modules.add1 import loadmap, compressArray, decompress, generateName, loadLAI from ..global_modules.settings import MaskInfo, LisSettings +from . import HydroModule -class evapowater(object): +class evapowater(HydroModule): """ # ************************************************************ # ***** EVAPORATION FROM OPEN WATER ************************** # ************************************************************ """ + input_files_keys = { + 'openwaterevapo': ['LakeMask', 'maxNoEva'], + 'varfractionwater': ['FracMaxWater', 'WFractionMaps'] + } + module_name = 'EvapoWater' def __init__(self, evapowater_variable): self.var = evapowater_variable diff --git a/src/lisflood/hydrological_modules/frost.py b/src/lisflood/hydrological_modules/frost.py index 4a15961f..e7237f36 100644 --- a/src/lisflood/hydrological_modules/frost.py +++ b/src/lisflood/hydrological_modules/frost.py @@ -20,9 +20,10 @@ import numpy as np from ..global_modules.add1 import loadmap +from . import HydroModule -class frost(object): +class frost(HydroModule): """ # ************************************************************ @@ -30,6 +31,8 @@ class frost(object): # ************************************************************ # Domain: whole pixel (permeable + direct runoff areas) """ + input_files_keys = {'all': ['Kfrost', 'Afrost', 'FrostIndexThreshold', 'SnowWaterEquivalent', 'FrostIndexInitValue']} + module_name = 'Frost' def __init__(self, frost_variable): self.var = frost_variable diff --git a/src/lisflood/hydrological_modules/groundwater.py b/src/lisflood/hydrological_modules/groundwater.py index 9aa61ebc..f9d9955f 100644 --- a/src/lisflood/hydrological_modules/groundwater.py +++ b/src/lisflood/hydrological_modules/groundwater.py @@ -22,14 +22,18 @@ from ..global_modules.add1 import loadmap, makenumpy, defsoil from ..global_modules.settings import LisSettings, MaskInfo from ..global_modules.errors import LisfloodError +from . import HydroModule -class groundwater(object): +class groundwater(HydroModule): """ # ************************************************************ # ***** GROUNDWATER ***************************************** # ************************************************************ """ + input_files_keys = {'all': ['UpperZoneTimeConstant', 'LowerZoneTimeConstant', 'LZInitValue', + 'LZThreshold', 'UZInitValue', 'UZForestInitValue', 'UZIrrigationInitValue']} + module_name = 'GroundWater' def __init__(self, groundwater_variable): self.var = groundwater_variable diff --git a/src/lisflood/hydrological_modules/indicatorcalc.py b/src/lisflood/hydrological_modules/indicatorcalc.py index c4c0d64c..bebe2cd8 100644 --- a/src/lisflood/hydrological_modules/indicatorcalc.py +++ b/src/lisflood/hydrological_modules/indicatorcalc.py @@ -24,15 +24,19 @@ from ..global_modules.add1 import loadmap, readnetcdf, decompress, compressArray from ..global_modules.settings import LisSettings, MaskInfo +from . import HydroModule -class indicatorcalc(object): +class indicatorcalc(HydroModule): """ # ************************************************************ # ***** Indicator calculation ************************************ # ************************************************************ """ + input_files_keys = {'indicator': ['Population', 'LandUseMask'], + 'TransientLandUseChange': ['PopulationMaps']} + module_name = 'IndicatorCalculation' def __init__(self, indicatorcalc_variable): self.var = indicatorcalc_variable @@ -97,9 +101,9 @@ def dynamic(self): self.var.monthend = next_date_time.month != self.var.CalendarDate.month self.var.yearend = next_date_time.year != self.var.CalendarDate.year # sum up every day - self.var.DayCounter = self.var.DayCounter + 1.0 - self.var.MonthETpotMM = self.var.MonthETpotMM + self.var.ETRef - self.var.MonthETactMM = self.var.MonthETactMM + self.var.deffraction(self.var.TaInterception) + self.var.TaPixel + self.var.ESActPixel + self.var.DayCounter = self.var.DayCounter + 1.0 + self.var.MonthETpotMM = self.var.MonthETpotMM + self.var.ETRef + self.var.MonthETactMM = self.var.MonthETactMM + self.var.deffraction(self.var.TaInterception) + self.var.TaPixel + self.var.ESActPixel if option['openwaterevapo']: self.var.MonthETactMM += self.var.EvaAddM3 * self.var.M3toMM diff --git a/src/lisflood/hydrological_modules/inflow.py b/src/lisflood/hydrological_modules/inflow.py index 00e16080..efada4ac 100644 --- a/src/lisflood/hydrological_modules/inflow.py +++ b/src/lisflood/hydrological_modules/inflow.py @@ -23,9 +23,10 @@ from ..global_modules.settings import LisSettings from ..global_modules.add1 import loadmap, read_tss_header, compressArray from ..global_modules.errors import LisfloodWarning +from . import HydroModule -class inflow(object): +class inflow(HydroModule): """ # ************************************************************ @@ -34,6 +35,8 @@ class inflow(object): # If option "inflow" is set to 1 the inflow hydrograph code is used # otherwise dummy code is used """ + input_files_keys = {'inflow': ['InflowPoints', 'QInTS']} + module_name = 'InFlow' def __init__(self, inflow_variable): self.var = inflow_variable diff --git a/src/lisflood/hydrological_modules/kinematic_wave_parallel.py b/src/lisflood/hydrological_modules/kinematic_wave_parallel.py index 3dd0e121..c934269d 100644 --- a/src/lisflood/hydrological_modules/kinematic_wave_parallel.py +++ b/src/lisflood/hydrological_modules/kinematic_wave_parallel.py @@ -21,7 +21,7 @@ pixelsm in previous sets have already been routed, pixels can be routed independently of each other and thus in parallel. For further details, see the method kinematicWave._setRoutingOrders. """ -from __future__ import absolute_import, division, print_function +from __future__ import absolute_import, division, print_function, unicode_literals __author__ = "Emiliano Gelati" __contact__ = "emiliano.gelati@ec.europa.eu" @@ -29,29 +29,45 @@ __date__ = "2016/06/01" import os +import glob + import numpy as np import pandas as pd import numexpr as nx from multiprocessing import cpu_count from platform import system -# IMPORT THE PARALLELISE KINEMATIC WAVE RUTING MODULE: IF IT WAS NOT COMPILED ON THE CURRENT MACHINE, ROUTING IS EXECUTED SERIALLY -# If the binary .so file does not exist or is not newer than the source .pyx, then the Cython module is imported directly from the source. -# For safety against binary files compiled on other machines and copied here, the binary must be at least 10 seconds younger than the source (see line 13). -# Importing directly from the source prevents using OpenMP multithreading. In such case, the routing is executed serially. +from ..global_modules.errors import LisfloodWarning + +# IMPORT THE PARALLELISE KINEMATIC WAVE RUTING MODULE: IF IT WAS NOT COMPILED ON THE CURRENT MACHINE, +# ROUTING IS EXECUTED SERIALLY if the binary .so file does not exist or is not newer than the source .pyx, +# then the Cython module is imported directly from the source. +# For safety against binary files compiled on other machines and copied here, +# the binary must be at least 10 seconds younger than the source (see line 13). +# Importing directly from the source prevents using OpenMP multithreading. +# In such case, the routing is executed serially. WINDOWS_OS = system() == "Windows" ROOT = os.path.join(os.path.dirname(os.path.realpath(__file__)), "kinematic_wave_parallel_tools") -SRC = ROOT + ".pyx" -BIN = ROOT + (".pyd" if WINDOWS_OS else ".so") - - -if not os.path.exists(BIN) or (os.path.exists(BIN) and os.path.exists(SRC) and os.stat(BIN).st_mtime < os.stat(SRC).st_mtime): - import pyximport # Activate the direct import from source of Cython modules. - setup_args = {"script_args": ["--compiler=mingw32"]} if WINDOWS_OS else None # Extra compiler argument under Windows - pyximport.install(setup_args=setup_args) # If this is executed, the binary .so file will be ignored and the routing is executed serially. - print("""WARNING:\nThe Cython module {} has not been compiled on the current machine (to compile, see instructions in the module's docstring). -The kinematic wave routing is executed serially (not in parallel).""".format(SRC)) +SRC = '{}.pyx'.format(ROOT) +BINS = '{}*.so'.format(ROOT) if not WINDOWS_OS else '{}*.pyd'.format(ROOT) + +bins = glob.glob(BINS) + +older = False +for binary in bins: + if os.stat(binary).st_mtime < os.stat(SRC).st_mtime: + older = True + break + +if not bins or older: + # Activate the direct import from source of Cython modules. + import pyximport + setup_args = {"script_args": ["--compiler=mingw32"]} if WINDOWS_OS else None + # If this is executed, the binary .so file will be ignored and the routing is executed serially. + pyximport.install(setup_args=setup_args) + print(LisfloodWarning("""The Cython module {} has not been compiled on the current machine (to compile, see instructions in the module's docstring). +The kinematic wave routing is executed serially (not in parallel).""".format(SRC))) from . import kinematic_wave_parallel_tools as kwpt diff --git a/src/lisflood/hydrological_modules/lakes.py b/src/lisflood/hydrological_modules/lakes.py index 81f9571c..be3f40fe 100644 --- a/src/lisflood/hydrological_modules/lakes.py +++ b/src/lisflood/hydrological_modules/lakes.py @@ -22,15 +22,19 @@ from ..global_modules.errors import LisfloodWarning from ..global_modules.add1 import loadmap, compressArray, decompress from ..global_modules.settings import LisSettings, MaskInfo +from . import HydroModule -class lakes(object): +class lakes(HydroModule): """ # ************************************************************ # ***** LAKES ***************************************** # ************************************************************ """ + input_files_keys = {'simulateLakes': ['LakeSites', 'TabLakeArea', 'TabLakeA', 'LakeMultiplier', + 'LakeInitialLevelValue', 'TabLakeAvNetInflowEstimate', 'PrevDischarge']} + module_name = 'Lakes' def __init__(self, lakes_variable): self.var = lakes_variable @@ -53,14 +57,12 @@ def initial(self): if option['simulateLakes']: LakeSitesC = loadmap('LakeSites') - LakeSitesC[LakeSitesC < 1] = 0 LakeSitesC[self.var.IsChannel == 0] = 0 - # self.var.LakeSites = ifthen((defined(self.var.LakeSites) & boolean(decompress(self.var.IsChannel))), self.var.LakeSites) # Get rid of any lakes that are not part of the channel network # mask lakes sites when using sub-catchments mask - self.var.LakeSitesCC = np.compress(LakeSitesC>0,LakeSitesC) + self.var.LakeSitesCC = np.compress(LakeSitesC > 0, LakeSitesC) self.var.LakeIndex = np.nonzero(LakeSitesC)[0] if self.var.LakeSitesCC.size == 0: @@ -70,7 +72,6 @@ def initial(self): # break if no lakes self.var.IsStructureKinematic = np.where(LakeSitesC > 0, np.bool8(1), self.var.IsStructureKinematic) - #self.var.IsStructureKinematic = ifthenelse(defined(self.var.LakeSites), pcraster.boolean(1), self.var.IsStructureKinematic) # Add lake locations to structures map (used to modify LddKinematic # and to calculate LddStructuresKinematic) @@ -84,14 +85,13 @@ def initial(self): # Get all pixels just upstream of lakes #----------------------- - #self.var.LakeInflowOld = cover(ifthen(defined(self.var.LakeSites), upstream(self.var.LddKinematic, self.var.ChanQ)), scalar(0.0)) self.var.LakeInflowOldCC = np.bincount(self.var.downstruct, weights=self.var.ChanQ)[self.var.LakeIndex] # for Modified Puls Method the Q(inflow)1 has to be used. It is assumed that this is the same as Q(inflow)2 for the first timestep # has to be checked if this works in forecasting mode! LakeArea = pcraster.lookupscalar(str(binding['TabLakeArea']), LakeSitePcr) LakeAreaC = compressArray(LakeArea) - self.var.LakeAreaCC = np.compress(LakeSitesC>0,LakeAreaC) + self.var.LakeAreaCC = np.compress(LakeSitesC > 0, LakeAreaC) # Surface area of each lake [m2] LakeA = pcraster.lookupscalar(str(binding['TabLakeA']), LakeSitePcr) @@ -181,7 +181,6 @@ def dynamic_inloop(self, NoRoutingExecuted): option = settings.options maskinfo = MaskInfo.instance() if not(option['InitLisflood']) and option['simulateLakes']: # only with no InitLisflood - #self.var.LakeInflow = cover(ifthen(defined(self.var.LakeSites), upstream(self.var.LddStructuresKinematic, self.var.ChanQ)), scalar(0.0)) self.var.LakeInflowCC = np.bincount(self.var.downstruct, weights=self.var.ChanQ)[self.var.LakeIndex] # Lake inflow in [m3/s] @@ -205,7 +204,6 @@ def dynamic_inloop(self, NoRoutingExecuted): # solution of this quadratic equation: # Q=sqr(-LakeFactor+sqrt(sqr(LakeFactor)+2*SI)); - #self.var.QLakeOutM3Dt = cover(self.var.LakeOutflow * self.var.DtRouting, scalar(0.0)) QLakeOutM3DtCC = self.var.LakeOutflow * self.var.DtRouting # Outflow in [m3] per timestep # Needed at every cell, hence cover statement @@ -216,14 +214,12 @@ def dynamic_inloop(self, NoRoutingExecuted): # self.var.LakeStorageM3CC < 0 leads to NaN in state files # Check LakeStorageM3CC for negative values and set them to zero if any(np.isnan(self.var.LakeStorageM3CC)) or any(self.var.LakeStorageM3CC < 0): - msg = "Negative or NaN volume for lake storage set to 0. Increase computation time step for routing (DtSecChannel) \n" + msg = "Negative or NaN volume for lake storage set to 0. " \ + "Increase computation time step for routing (DtSecChannel) \n" print(LisfloodWarning(msg)) self.var.LakeStorageM3CC[self.var.LakeStorageM3CC < 0] = 0 self.var.LakeStorageM3CC[np.isnan(self.var.LakeStorageM3CC)] = 0 - #self.var.LakeStorageM3CC = cover(self.var.LakeStorageM3CC - self.var.EWLakeM3Dt, scalar(0.0)) - # New lake storage [m3] (assuming lake surface area equals bottom area) - #self.var.LakeStorageM3Balance += LakeIn * self.var.DtRouting - self.var.QLakeOutM3Dt - self.var.EWLakeM3Dt self.var.LakeStorageM3BalanceCC += LakeIn * self.var.DtRouting - QLakeOutM3DtCC # for mass balance, the lake storage is calculated every time step self.var.LakeLevelCC = self.var.LakeStorageM3CC / self.var.LakeAreaCC @@ -234,16 +230,17 @@ def dynamic_inloop(self, NoRoutingExecuted): if option['repsimulateLakes']: if NoRoutingExecuted == 0: - self.var.LakeInflowM3S = maskinfo.in_zero() - self.var.LakeOutflowM3S = maskinfo.in_zero() - self.var.sumLakeInCC = self.var.LakeInflowCC * self.var.DtRouting - self.var.sumLakeOutCC = QLakeOutM3DtCC - # for timeseries output - in and outflow to the reservoir is sumed up over the sub timesteps and stored in m/s - # set to zero at first timestep + self.var.LakeInflowM3S = maskinfo.in_zero() + self.var.LakeOutflowM3S = maskinfo.in_zero() + self.var.sumLakeInCC = self.var.LakeInflowCC * self.var.DtRouting + self.var.sumLakeOutCC = QLakeOutM3DtCC + # for timeseries output - in and outflow to the reservoir + # is sumed up over the sub timesteps and stored in m/s + # set to zero at first timestep else: - self.var.sumLakeInCC += self.var.LakeInflowCC * self.var.DtRouting - self.var.sumLakeOutCC += QLakeOutM3DtCC - # summing up over all sub timesteps + self.var.sumLakeInCC += self.var.LakeInflowCC * self.var.DtRouting + self.var.sumLakeOutCC += QLakeOutM3DtCC + # summing up over all sub timesteps if NoRoutingExecuted == (self.var.NoRoutSteps-1): @@ -251,10 +248,10 @@ def dynamic_inloop(self, NoRoutingExecuted): self.var.LakeStorageM3Balance = maskinfo.in_zero() self.var.LakeStorageM3 = maskinfo.in_zero() self.var.LakeLevel = maskinfo.in_zero() - np.put(self.var.LakeStorageM3Balance,self.var.LakeIndex,self.var.LakeStorageM3BalanceCC) - np.put(self.var.LakeStorageM3,self.var.LakeIndex,self.var.LakeStorageM3CC) - np.put(self.var.LakeLevel,self.var.LakeIndex,self.var.LakeLevelCC) + np.put(self.var.LakeStorageM3Balance, self.var.LakeIndex, self.var.LakeStorageM3BalanceCC) + np.put(self.var.LakeStorageM3, self.var.LakeIndex, self.var.LakeStorageM3CC) + np.put(self.var.LakeLevel, self.var.LakeIndex, self.var.LakeLevelCC) if option['repsimulateLakes']: - np.put(self.var.LakeInflowM3S ,self.var.LakeIndex,self.var.sumLakeInCC / self.var.DtSec) - np.put(self.var.LakeOutflowM3S,self.var.LakeIndex,self.var.sumLakeOutCC / self.var.DtSec) + np.put(self.var.LakeInflowM3S, self.var.LakeIndex, self.var.sumLakeInCC / self.var.DtSec) + np.put(self.var.LakeOutflowM3S, self.var.LakeIndex, self.var.sumLakeOutCC / self.var.DtSec) diff --git a/src/lisflood/hydrological_modules/landusechange.py b/src/lisflood/hydrological_modules/landusechange.py index 701849cf..33555d2b 100644 --- a/src/lisflood/hydrological_modules/landusechange.py +++ b/src/lisflood/hydrological_modules/landusechange.py @@ -18,9 +18,10 @@ from ..global_modules.add1 import loadmap, readnetcdf from ..global_modules.settings import LisSettings +from . import HydroModule -class landusechange(object): +class landusechange(HydroModule): """ # ************************************************************ @@ -35,6 +36,11 @@ class landusechange(object): # rice irrigation areas # other """ + input_files_keys = {'all': ['ForestFraction', 'DirectRunoffFraction', 'WaterFraction', + 'IrrigationFraction', 'RiceFraction', 'OtherFraction'], + 'TransientLandUseChange': ['ForestFractionMaps', 'DirectRunoffFractionMaps', 'WaterFractionMaps', + 'IrrigationFractionMaps', 'RiceFractionMaps', 'OtherFractionMaps']} + module_name = 'LandUseChange' def __init__(self, landusechange_variable): self.var = landusechange_variable @@ -46,12 +52,12 @@ def initial(self): """ initial part of the landusechange module """ - self.var.ForestFraction = loadmap('ForestFraction',timestampflag='closest') - self.var.DirectRunoffFraction = loadmap('DirectRunoffFraction',timestampflag='closest') - self.var.WaterFraction = loadmap('WaterFraction',timestampflag='closest') - self.var.IrrigationFraction = loadmap('IrrigationFraction',timestampflag='closest') - self.var.RiceFraction = loadmap('RiceFraction',timestampflag='closest') - self.var.OtherFraction = loadmap('OtherFraction',timestampflag='closest') + self.var.ForestFraction = loadmap('ForestFraction', timestampflag='closest') + self.var.DirectRunoffFraction = loadmap('DirectRunoffFraction', timestampflag='closest') + self.var.WaterFraction = loadmap('WaterFraction', timestampflag='closest') + self.var.IrrigationFraction = loadmap('IrrigationFraction', timestampflag='closest') + self.var.RiceFraction = loadmap('RiceFraction', timestampflag='closest') + self.var.OtherFraction = loadmap('OtherFraction', timestampflag='closest') def dynamic(self): """dynamic part of the landusechange module diff --git a/src/lisflood/hydrological_modules/leafarea.py b/src/lisflood/hydrological_modules/leafarea.py index 57655f34..f613df0d 100644 --- a/src/lisflood/hydrological_modules/leafarea.py +++ b/src/lisflood/hydrological_modules/leafarea.py @@ -21,15 +21,18 @@ from ..global_modules.add1 import loadmap, generateName, loadLAI from ..global_modules.settings import LisSettings +from . import HydroModule -class leafarea(object): +class leafarea(HydroModule): """ # ************************************************************ # ***** LEAF AREA INDEX DATA **************************** # ************************************************************ """ + input_files_keys = {'all': ['kdf', 'LAIOtherMaps', 'LAIForestMaps', 'LAIIrrigationMaps']} + module_name = 'LeafArea' def __init__(self, leafarea_variable): self.var = leafarea_variable diff --git a/src/lisflood/hydrological_modules/miscInitial.py b/src/lisflood/hydrological_modules/miscInitial.py index cd2f88ce..ab26edff 100644 --- a/src/lisflood/hydrological_modules/miscInitial.py +++ b/src/lisflood/hydrological_modules/miscInitial.py @@ -24,6 +24,7 @@ from ..global_modules.add1 import loadmap, compressArray from ..global_modules.settings import calendar, MaskAttrs, MaskInfo, LisSettings +from . import HydroModule def coordinatesLand(eastings_forcing, northings_forcing): @@ -35,14 +36,17 @@ def coordinatesLand(eastings_forcing, northings_forcing): row_slice = slice(top_row, top_row + maskattrs['row']) col_slice = slice(left_col, left_col + maskattrs['col']) maskinfo = MaskInfo.instance() - return [co[row_slice,col_slice][~maskinfo.info.mask] for co in np.meshgrid(eastings_forcing, northings_forcing)] + return [co[row_slice, col_slice][~maskinfo.info.mask] for co in np.meshgrid(eastings_forcing, northings_forcing)] -class miscInitial(object): +class miscInitial(HydroModule): """ Miscellaneous repeatedly used expressions """ + input_files_keys = {'all': ['DtSecChannel', 'DtSec', 'GwLoss', 'GwPercValue', 'PrScaling', 'CalEvaporation'], + 'gridSizeUserDefined': ['PixelLengthUser', 'PixelAreaUser']} + module_name = 'MiscInitial' def __init__(self, misc_variable): self.var = misc_variable @@ -68,11 +72,11 @@ def initial(self): # Limitation: always assumes square grid cells (not rectangles!). Size of grid cells # may vary across map though - self.var.PixelLengthPcr = loadmap('PixelLengthUser',pcr=True) + self.var.PixelLengthPcr = loadmap('PixelLengthUser', pcr=True) self.var.PixelLength = compressArray(self.var.PixelLengthPcr) # Length of pixel [m] # Area of pixel [m2] - self.var.PixelAreaPcr = loadmap('PixelAreaUser',pcr=True) + self.var.PixelAreaPcr = loadmap('PixelAreaUser', pcr=True) self.var.PixelArea = compressArray(self.var.PixelAreaPcr) else: diff --git a/src/lisflood/hydrological_modules/opensealed.py b/src/lisflood/hydrological_modules/opensealed.py index a1e7f3fa..8beaac9f 100644 --- a/src/lisflood/hydrological_modules/opensealed.py +++ b/src/lisflood/hydrological_modules/opensealed.py @@ -17,16 +17,20 @@ from __future__ import absolute_import, division import numpy as np + from ..global_modules.settings import MaskInfo +from . import HydroModule -class opensealed(object): +class opensealed(HydroModule): """ # ************************************************************ # ***** SOIL LOOP ***************************************** # ************************************************************ """ + input_files_keys = {'all': []} + module_name = 'OpenSealed' def __init__(self, opensealed_variable): self.var = opensealed_variable diff --git a/src/lisflood/hydrological_modules/polder.py b/src/lisflood/hydrological_modules/polder.py index 25c1b8eb..8d7846bf 100644 --- a/src/lisflood/hydrological_modules/polder.py +++ b/src/lisflood/hydrological_modules/polder.py @@ -21,15 +21,18 @@ from ..global_modules.add1 import loadmap from ..global_modules.settings import LisSettings +from . import HydroModule -class polder(object): +class polder(HydroModule): """ # ************************************************************ # ***** POLDER ***************************************** # ************************************************************ """ + input_files_keys = {'simulatePolders': ['PolderSites', 'TabPolderArea', 'PolderInitialLevelValue']} + module_name = 'Polder' def __init__(self, polder_variable): self.var = polder_variable @@ -58,7 +61,7 @@ def initial(self): # Flag that is boolean(1) for polder sites and boolean(0) otherwise # total storage capacity of Polder area [m3] - PolderArea = pcraster.lookupscalar(binding['TabPolderArea'], PolderSites) + PolderArea = pcraster.lookupscalar(str(binding['TabPolderArea']), PolderSites) PolderLevel = binding['PolderInitialLevelValue'] # Initial polder level [m] self.var.PolderStorageIniM3 = pcraster.cover(PolderLevel * PolderArea, pcraster.scalar(0.0)) @@ -71,6 +74,7 @@ def dynamic_init(): """ dynamic part of the polder module initialising polders """ + pass # ************************************************************ # ***** POLDER diff --git a/src/lisflood/hydrological_modules/readmeteo.py b/src/lisflood/hydrological_modules/readmeteo.py index 7b424249..a64af0c1 100644 --- a/src/lisflood/hydrological_modules/readmeteo.py +++ b/src/lisflood/hydrological_modules/readmeteo.py @@ -35,10 +35,10 @@ def __init__(self, readmeteo_variable): binding = settings.binding if option['readNetcdfStack']: # checking if time period in netCDF files (forcings) includes simulation period - checknetcdf(binding['PrecipitationMaps'], binding['StepStart'], binding['StepEnd'] ) - checknetcdf(binding['TavgMaps'], binding['StepStart'], binding['StepEnd'] ) - checknetcdf(binding['ET0Maps'], binding['StepStart'], binding['StepEnd'] ) - checknetcdf(binding['E0Maps'], binding['StepStart'], binding['StepEnd'] ) + checknetcdf(binding['PrecipitationMaps'], binding['StepStart'], binding['StepEnd']) + checknetcdf(binding['TavgMaps'], binding['StepStart'], binding['StepEnd']) + checknetcdf(binding['ET0Maps'], binding['StepStart'], binding['StepEnd']) + checknetcdf(binding['E0Maps'], binding['StepStart'], binding['StepEnd']) # -------------------------------------------------------------------------- # -------------------------------------------------------------------------- @@ -59,7 +59,6 @@ def dynamic(self): self.var.Precipitation = readnetcdf(binding['PrecipitationMaps'], self.var.currentTimeStep()) * self.var.DtDay * self.var.PrScaling self.var.Tavg = readnetcdf(binding['TavgMaps'], self.var.currentTimeStep()) self.var.ETRef = readnetcdf(binding['ET0Maps'], self.var.currentTimeStep()) * self.var.DtDay * self.var.CalEvaporation - # self.var.ESRef = readnetcdf(binding['ES0Maps'], self.var.currentTimeStep()) * self.var.DtDay * self.var.CalEvaporation self.var.EWRef = readnetcdf(binding['E0Maps'], self.var.currentTimeStep()) * self.var.DtDay * self.var.CalEvaporation else: # Read from stack of maps in Pcraster format @@ -69,7 +68,6 @@ def dynamic(self): # average DAILY temperature (even if you are running the model on say an hourly time step) [degrees C] self.var.ETRef = readmapsparse(binding['ET0Maps'], self.var.currentTimeStep(), self.var.ETRef) * self.var.DtDay * self.var.CalEvaporation # daily reference evapotranspiration (conversion to [mm] per time step) - # self.var.ESRef = readmapsparse(binding['ES0Maps'], self.var.currentTimeStep(), self.var.ESRef) * self.var.DtDay * self.var.CalEvaporation # potential evaporation rate from a bare soil surface (conversion to [mm] per time step) self.var.EWRef = readmapsparse(binding['E0Maps'], self.var.currentTimeStep(), self.var.EWRef) * self.var.DtDay * self.var.CalEvaporation # potential evaporation rate from water surface (conversion to [mm] per time step) diff --git a/src/lisflood/hydrological_modules/reservoir.py b/src/lisflood/hydrological_modules/reservoir.py index fff4eb64..b5f8647f 100644 --- a/src/lisflood/hydrological_modules/reservoir.py +++ b/src/lisflood/hydrological_modules/reservoir.py @@ -2,14 +2,16 @@ Copyright 2019 European Union -Licensed under the EUPL, Version 1.2 or as soon they will be approved by the European Commission subsequent versions of the EUPL (the "Licence"); +Licensed under the EUPL, Version 1.2 or as soon they will be approved by the European Commission +subsequent versions of the EUPL (the "Licence"); You may not use this work except in compliance with the Licence. You may obtain a copy of the Licence at: https://joinup.ec.europa.eu/sites/default/files/inline-files/EUPL%20v1_2%20EN(1).txt -Unless required by applicable law or agreed to in writing, software distributed under the Licence is distributed on an "AS IS" basis, +Unless required by applicable law or agreed to in writing, +software distributed under the Licence is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the Licence for the specific language governing permissions and limitations under the Licence. @@ -23,15 +25,21 @@ from ..global_modules.settings import LisSettings, MaskInfo from ..global_modules.add1 import loadmap, compressArray, decompress, makenumpy from ..global_modules.errors import LisfloodWarning +from . import HydroModule -class reservoir(object): +class reservoir(HydroModule): """ # ************************************************************ # ***** RESERVOIR ***************************************** # ************************************************************ """ + input_files_keys = {'simulateReservoirs': ['ReservoirSites', 'TabTotStorage', 'TabConservativeStorageLimit', + 'TabNormalStorageLimit', 'TabFloodStorageLimit', 'TabNonDamagingOutflowQ', + 'TabNormalOutflowQ', 'TabMinOutflowQ', 'adjust_Normal_Flood', + 'ReservoirRnormqMult', 'ReservoirInitialFillValue']} + module_name = 'Reservoir' def __init__(self, reservoir_variable): self.var = reservoir_variable diff --git a/src/lisflood/hydrological_modules/riceirrigation.py b/src/lisflood/hydrological_modules/riceirrigation.py index affe5165..05fe631d 100644 --- a/src/lisflood/hydrological_modules/riceirrigation.py +++ b/src/lisflood/hydrological_modules/riceirrigation.py @@ -20,14 +20,18 @@ from ..global_modules.add1 import loadmap from ..global_modules.settings import MaskInfo, LisSettings +from . import HydroModule -class riceirrigation(object): +class riceirrigation(HydroModule): """ # ************************************************************ # ***** Rice irrigation ************************************ # ************************************************************ """ + input_files_keys = {'riceIrrigation': ['RiceFlooding', 'RicePercolation', 'RicePlantingDay1', + 'RiceHarvestDay1', 'RicePlantingDay2', 'RiceHarvestDay2']} + module_name = 'RiceIrrigation' def __init__(self, riceirrigation_variable): self.var = riceirrigation_variable diff --git a/src/lisflood/hydrological_modules/routing.py b/src/lisflood/hydrological_modules/routing.py index 89e1483e..4951d790 100644 --- a/src/lisflood/hydrological_modules/routing.py +++ b/src/lisflood/hydrological_modules/routing.py @@ -30,15 +30,22 @@ from ..global_modules.settings import LisSettings, MaskInfo from ..global_modules.add1 import loadmap, compressArray, decompress +from . import HydroModule -class routing(object): +class routing(HydroModule): """ # ************************************************************ # ***** ROUTING ***************************************** # ************************************************************ """ + input_files_keys = {'all': ['beta', 'ChanLength', 'Ldd', 'Channels', 'ChanGrad', 'ChanGradMin', + 'CalChanMan', 'ChanMan', 'ChanBottomWidth', 'ChanDepthThreshold', + 'ChanSdXdY', 'TotalCrossSectionAreaInitValue', 'PrevDischarge'], + 'SplitRouting': ['CrossSection2AreaInitValue', 'PrevSideflowInitValue', 'CalChanMan2'], + 'dynamicWave': ['ChannelsDynamic']} + module_name = 'Routing' def __init__(self, routing_variable): self.var = routing_variable @@ -80,7 +87,7 @@ def initial(self): # -------------------------- LDD - self.var.Ldd = lddmask(loadmap('Ldd',pcr=True,lddflag=True), self.var.MaskMap) + self.var.Ldd = lddmask(loadmap('Ldd', pcr=True, lddflag=True), self.var.MaskMap) # Cut ldd to size of MaskMap (NEW, 29/9/2004) # Prevents 'unsound' ldd if MaskMap covers sub-area of ldd @@ -97,13 +104,12 @@ def initial(self): self.var.InvUpArea = 1 / self.var.UpArea # Calculate inverse, so we can multiply in dynamic (faster than divide) - self.var.IsChannelPcr = boolean(loadmap('Channels',pcr=True)) + self.var.IsChannelPcr = boolean(loadmap('Channels', pcr=True)) self.var.IsChannel = np.bool8(compressArray(self.var.IsChannelPcr)) # Identify channel pixels self.var.IsChannelKinematic = self.var.IsChannel.copy() # Identify kinematic wave channel pixels # (identical to IsChannel, unless dynamic wave is used, see below) - #self.var.IsStructureKinematic = pcraster.boolean(0) self.var.IsStructureKinematic = np.bool8(maskinfo.in_zero()) # Map that identifies special inflow/outflow structures (reservoirs, lakes) within the @@ -122,29 +128,15 @@ def initial(self): # find outlet points... if option['dynamicWave']: - IsChannelDynamic = boolean(loadmap('ChannelsDynamic',pcr=True)) + IsChannelDynamic = boolean(loadmap('ChannelsDynamic', pcr=True)) # Identify channel pixels where dynamic wave is used self.var.IsChannelKinematic = (self.var.IsChannelPcr == 1) & (IsChannelDynamic == 0) # Identify (update) channel pixels where kinematic wave is used self.var.LddKinematic = lddmask(self.var.Ldd, self.var.IsChannelKinematic) # Ldd for kinematic wave: ends (pit) just before dynamic stretch - LddDynamic = lddmask(self.var.Ldd, IsChannelDynamic) - # Ldd for dynamic wave # Following statements produce an ldd network that connects the pits in # LddKinematic to the nearest downstream dynamic wave pixel - LddToDyn = lddrepair(ifthenelse(IsChannelDynamic, 5, self.var.Ldd)) - # Temporary ldd: flow paths end in dynamic pixels - PitsKinematic = cover(boolean(pit(self.var.LddKinematic)), 0) - # Define start of each flow path at pit on LddKinematic - PathKinToDyn = path(LddToDyn, PitsKinematic) - # Identify paths that connect pits in LddKinematic to dynamic wave - # pixels - LddKinToDyn = lddmask(LddToDyn, PathKinToDyn) - # Create ldd - DynWaveBoundaryCondition = boolean(pit(LddDynamic)) - # NEW 12-7-2005 (experimental) - # Location of boundary condition dynamic wave self.var.AtLastPoint = (downstream(self.var.Ldd, AtOutflow) == 1) & (AtOutflow != 1) & self.var.IsChannelPcr @@ -165,23 +157,19 @@ def initial(self): # assign unique identifier to each of them maskinfo = MaskInfo.instance() lddC = compressArray(self.var.LddKinematic) - inAr = decompress(np.arange(maskinfo.info.mapC[0],dtype="int32")) + inAr = decompress(np.arange(maskinfo.info.mapC[0], dtype="int32")) # giving a number to each non missing pixel as id - self.var.downstruct = (compressArray(downstream(self.var.LddKinematic,inAr))).astype("int32") + self.var.downstruct = (compressArray(downstream(self.var.LddKinematic, inAr))).astype("int32") # each upstream pixel gets the id of the downstream pixel self.var.downstruct[lddC == 5] = maskinfo.info.mapC[0] # all pits gets a high number - #d3=np.bincount(self.var.down, weights=loadmap('AvgDis'))[:-1] # upstream function in numpy OutflowPoints = nominal(uniqueid(self.var.AtLastPoint)) - # and assign unique identifier to each of them + # and assign unique identifier to each of them self.var.Catchments = (compressArray(catchment(self.var.Ldd, OutflowPoints))).astype(np.int32) CatchArea = np.bincount(self.var.Catchments, weights=self.var.PixelArea)[self.var.Catchments] - #CatchArea = CatchArea[self.var.Catchments] # define catchment for each outflow point - #CatchArea = areatotal(self.var.PixelArea, self.var.Catchments) - # Compute area of each catchment [m2] # Note: in earlier versions this was calculated using the "areaarea" function, # changed to "areatotal" in order to enable handling of grids with spatially @@ -396,15 +384,15 @@ def initialSecond(self): self.var.Chan2QStart = self.var.QLimit - compressArray(upstream(self.var.LddKinematic, decompress(self.var.QLimit))) # because kinematic routing with a low amount of discharge leads to long travel time: # Starting Q for second line is set to a higher value - self.var.Chan2M3Kin = np.maximum(self.var.CrossSection2Area * self.var.ChanLength + self.var.Chan2M3Start, 0) # TEMPORARY SOLUTION TO VIRTUAL WATER PROBLEM - self.var.ChanM3Kin = np.maximum(self.var.ChanM3 - self.var.Chan2M3Kin + self.var.Chan2M3Start, 0) # TEMPORARY SOLUTION TO VIRTUAL WATER PROBLEM + self.var.Chan2M3Kin = np.maximum(self.var.CrossSection2Area * self.var.ChanLength + self.var.Chan2M3Start, 0) # TEMPORARY SOLUTION TO VIRTUAL WATER PROBLEM + self.var.ChanM3Kin = np.maximum(self.var.ChanM3 - self.var.Chan2M3Kin + self.var.Chan2M3Start, 0) # TEMPORARY SOLUTION TO VIRTUAL WATER PROBLEM self.var.Chan2QKin = (self.var.Chan2M3Kin*self.var.InvChanLength*self.var.InvChannelAlpha2)**(self.var.InvBeta) self.var.ChanQKin = (self.var.ChanM3Kin*self.var.InvChanLength*self.var.InvChannelAlpha)**(self.var.InvBeta) # Initialise parallel kinematic wave router: main channel-only routing if self.var.ChannelAlpha2 is None; else split-routing(main channel + floodplains) maskinfo = MaskInfo.instance() - self.river_router = kinematicWave(compressArray(self.var.LddKinematic), ~maskinfo.info.mask, self.var.ChannelAlpha,\ - self.var.Beta, self.var.ChanLength, self.var.DtRouting,\ + self.river_router = kinematicWave(compressArray(self.var.LddKinematic), ~maskinfo.info.mask, self.var.ChannelAlpha, + self.var.Beta, self.var.ChanLength, self.var.DtRouting, int(binding["numCPUs_parallelKinematicWave"]), alpha_floodplains=self.var.ChannelAlpha2) # -------------------------------------------------------------------------- diff --git a/src/lisflood/hydrological_modules/snow.py b/src/lisflood/hydrological_modules/snow.py index 13dba3f7..218b582e 100644 --- a/src/lisflood/hydrological_modules/snow.py +++ b/src/lisflood/hydrological_modules/snow.py @@ -21,9 +21,10 @@ from ..global_modules.add1 import loadmap from ..global_modules.settings import MaskInfo +from . import HydroModule -class snow(object): +class snow(HydroModule): """ # ************************************************************ @@ -39,6 +40,10 @@ class snow(object): # Zone B: center third # Zone C: upper third """ + input_files_keys = {'all': ['ElevationStD', 'TemperatureLapseRate', 'SnowSeasonAdj', + 'TempSnow', 'SnowFactor', 'SnowMeltCoef', 'TempMelt', + 'SnowCoverAInitValue', 'SnowCoverBInitValue', 'SnowCoverCInitValue']} + module_name = 'Snow' def __init__(self, snow_variable): self.var = snow_variable @@ -153,7 +158,6 @@ def dynamic(self): # if it's snowing then no rain SnowMeltS = (TavgS - self.var.TempMelt) * SeasSnowMeltCoef * (1 + 0.01 * RainS) * self.var.DtDay - if i < 2: IceMeltS = self.var.Tavg * 7.0 * self.var.DtDay * SummerSeason # if i = 0 and 1 -> higher and middle zone diff --git a/src/lisflood/hydrological_modules/soil.py b/src/lisflood/hydrological_modules/soil.py index 88343b18..62726153 100644 --- a/src/lisflood/hydrological_modules/soil.py +++ b/src/lisflood/hydrological_modules/soil.py @@ -21,15 +21,35 @@ from ..global_modules.settings import MaskInfo, LisSettings from ..global_modules.add1 import loadmap, defsoil +from . import HydroModule -class soil(object): +class soil(HydroModule): """ # ************************************************************ # ***** SOIL ***************************************** # ************************************************************ """ + input_files_keys = {'all': ['SoilDepth1', 'SoilDepth1Forest', 'SoilDepth2', 'SoilDepth2Forest', + 'SoilDepth3', 'SoilDepth3Forest', 'CourantCrit', 'LeafDrainageTimeConstant', + 'AvWaterRateThreshold', 'MapCropCoef', 'MapForestCropCoef', 'MapIrrigationCropCoef', + 'MapCropGroupNumber', 'MapForestCropGroupNumber', 'MapIrrigationCropGroupNumber', + 'MapN', 'MapForestN', 'MapKSat1', 'MapKSat1Forest', 'MapKSat2', 'MapKSat2Forest', 'MapKSat3', + 'MapLambda1', 'MapLambda1Forest', 'MapLambda2', 'MapLambda2Forest', 'MapLambda3', + 'MapGenuAlpha1', 'MapGenuAlpha1Forest', 'MapGenuAlpha2', 'MapGenuAlpha2Forest', 'MapGenuAlpha3', + 'MapThetaSat1', 'MapThetaSat1Forest', 'MapThetaSat2', 'MapThetaSat2Forest', 'MapThetaSat3', + 'MapThetaRes1', 'MapThetaRes1Forest', 'MapThetaRes2', 'MapThetaRes2Forest', 'MapThetaRes3', + 'ThetaInit1Value', 'ThetaForestInit1Value', 'ThetaIrrigationInit1Value', + 'ThetaInit2Value', 'ThetaForestInit2Value', 'ThetaIrrigationInit2Value', + 'ThetaInit3Value', 'ThetaForestInit3Value', 'ThetaIrrigationInit3Value', + 'b_Xinanjiang', 'PowerPrefFlow', + 'DSLRInitValue', 'DSLRForestInitValue', 'DSLRIrrigationInitValue', + 'CumIntInitValue', 'CumIntForestInitValue', 'CumIntIrrigationInitValue', + 'CumIntSealedInitValue', 'SMaxSealed'], + 'drainedIrrigation': ['DrainedFraction'], + 'simulatePF': ['HeadMax']} + module_name = 'Soil' def __init__(self, soil_variable): self.var = soil_variable @@ -65,7 +85,7 @@ def splitlanduse(array1, array2=None, array3=None): self.var.ForestFractionBase = self.var.ForestFraction.copy() self.var.DirectRunoffFractionBase = self.var.DirectRunoffFraction.copy() - self.var.SoilFraction = self.var.ForestFraction + self.var.OtherFraction + self.var.IrrigationFraction + self.var.SoilFraction = self.var.ForestFraction + self.var.OtherFraction + self.var.IrrigationFraction self.var.PermeableFraction = 1 - self.var.DirectRunoffFraction - self.var.WaterFraction # Permeable fraction of pixel @@ -73,9 +93,9 @@ def splitlanduse(array1, array2=None, array3=None): # soil depth top layer for every landuse but forest, impervious soil # and water - self.var.SoilDepth1a = defsoil('SoilDepth1','SoilDepth1Forest') - self.var.SoilDepth1b = defsoil('SoilDepth2','SoilDepth2Forest') - self.var.SoilDepth2 = defsoil('SoilDepth3','SoilDepth3Forest') + self.var.SoilDepth1a = defsoil('SoilDepth1', 'SoilDepth1Forest') + self.var.SoilDepth1b = defsoil('SoilDepth2', 'SoilDepth2Forest') + self.var.SoilDepth2 = defsoil('SoilDepth3', 'SoilDepth3Forest') # ----------------- miscParameters --------------------------- @@ -98,19 +118,18 @@ def splitlanduse(array1, array2=None, array3=None): # using maps instead of table to be more sensitive to landuse change # Soil properties for percentage of pixel area without forest, water , # impervious soil - self.var.CropCoef = defsoil('MapCropCoef','MapForestCropCoef','MapIrrigationCropCoef') + self.var.CropCoef = defsoil('MapCropCoef', 'MapForestCropCoef', 'MapIrrigationCropCoef') # crop coefficients for each land use class - self.var.CropGroupNumber = defsoil('MapCropGroupNumber','MapForestCropGroupNumber','MapIrrigationCropGroupNumber') + self.var.CropGroupNumber = defsoil('MapCropGroupNumber', 'MapForestCropGroupNumber', 'MapIrrigationCropGroupNumber') # crop group numbers for soil water depletion factor # self.var.NManning = [loadmap('MapN'), loadmap('MapForestN'), 0.02] - self.var.NManning = defsoil('MapN','MapForestN', 0.02) + self.var.NManning = defsoil('MapN', 'MapForestN', 0.02) # Manning's roughness coefficient for each land use class # third manning is from direct runoff # self.var.NManningDirect=scalar(0.02) - # ************************************************************ # ***** MAPS DERIVED FROM SOIL TEXTURE AND SOIL DEPTH ******** # ************************************************************ @@ -125,8 +144,8 @@ def splitlanduse(array1, array2=None, array3=None): Lambda1b = defsoil('MapLambda2', 'MapLambda2Forest') Lambda2 = defsoil('MapLambda3') # Pore-size index (for Van Genuchten parameters) - GenuAlpha1a = defsoil('MapGenuAlpha1','MapGenuAlpha1Forest') - GenuAlpha1b = defsoil('MapGenuAlpha2','MapGenuAlpha2Forest') + GenuAlpha1a = defsoil('MapGenuAlpha1', 'MapGenuAlpha1Forest') + GenuAlpha1b = defsoil('MapGenuAlpha2', 'MapGenuAlpha2Forest') GenuAlpha2 = defsoil('MapGenuAlpha3') # Van Genuchten Alpha coefficient ThetaS1a = defsoil('MapThetaSat1','MapThetaSat1Forest') @@ -222,9 +241,9 @@ def splitlanduse(array1, array2=None, array3=None): # ***** INITIAL VALUES for soil - ThetaInit1aValue = defsoil('ThetaInit1Value','ThetaForestInit1Value','ThetaIrrigationInit1Value') - ThetaInit1bValue = defsoil('ThetaInit2Value','ThetaForestInit2Value','ThetaIrrigationInit2Value') - ThetaInit2Value = defsoil('ThetaInit3Value','ThetaForestInit3Value','ThetaIrrigationInit3Value') + ThetaInit1aValue = defsoil('ThetaInit1Value', 'ThetaForestInit1Value', 'ThetaIrrigationInit1Value') + ThetaInit1bValue = defsoil('ThetaInit2Value', 'ThetaForestInit2Value', 'ThetaIrrigationInit2Value') + ThetaInit2Value = defsoil('ThetaInit3Value', 'ThetaForestInit3Value', 'ThetaIrrigationInit3Value') WInit1a = splitlanduse(maskinfo.in_zero()) WInit1b = splitlanduse(maskinfo.in_zero()) diff --git a/src/lisflood/hydrological_modules/soilloop.py b/src/lisflood/hydrological_modules/soilloop.py index 3ddc733b..36454667 100644 --- a/src/lisflood/hydrological_modules/soilloop.py +++ b/src/lisflood/hydrological_modules/soilloop.py @@ -19,16 +19,19 @@ import numpy as np -from lisflood.global_modules.settings import LisSettings, MaskInfo +from ..global_modules.settings import LisSettings, MaskInfo +from . import HydroModule -class soilloop(object): +class soilloop(HydroModule): """ # ************************************************************ # ***** SOIL LOOP ***************************************** # ************************************************************ """ + input_files_keys = {'wateruse': []} + module_name = 'SoilLoop' def __init__(self, soilloop_variable): self.var = soilloop_variable @@ -128,8 +131,8 @@ def dynamic(self, sLoop): settings = LisSettings.instance() option = settings.options if option['wateruse'] and sLoop == 2: - self.var.WFilla = np.minimum(WCrit1a,self.var.WPF3a[2]) - self.var.WFillb = np.minimum(WCrit1b,self.var.WPF3b[2]) + self.var.WFilla = np.minimum(WCrit1a, self.var.WPF3a[2]) + self.var.WFillb = np.minimum(WCrit1b, self.var.WPF3b[2]) # if water use is calculated, get the filling of the soil layer for either pF3 or WCrit1 # that is the amount of water the soil gets filled by water from irrigation @@ -451,7 +454,7 @@ def dynamic(self, sLoop): self.var.pF0[sLoop] = np.where(Head1a > 0, np.log10(Head1a), -1) self.var.pF1[sLoop] = np.where(Head1b > 0, np.log10(Head1b), -1) - self.var.pF2[sLoop] = np.where(Head2 > 0, np.log10(Head2) , -1) + self.var.pF2[sLoop] = np.where(Head2 > 0, np.log10(Head2), -1) # Compute pF. Set to -1 should heads become equal to mor less than 0. No idea # if this can even actually happen (copied this from old LISFLOOD version) but it # shouldn't do any harm. diff --git a/src/lisflood/hydrological_modules/surface_routing.py b/src/lisflood/hydrological_modules/surface_routing.py index 5b2257c7..9e6b98cd 100644 --- a/src/lisflood/hydrological_modules/surface_routing.py +++ b/src/lisflood/hydrological_modules/surface_routing.py @@ -21,15 +21,19 @@ from ..global_modules.add1 import loadmap, makenumpy, compressArray from ..global_modules.settings import MaskInfo, LisSettings from .kinematic_wave_parallel import kinematicWave +from . import HydroModule -class surface_routing(object): +class surface_routing(HydroModule): """ # ************************************************************ # ***** SURFACE ROUTING ************************************** # ************************************************************ """ + input_files_keys = {'all': ['OFOtherInitValue', 'OFForestInitValue', 'OFDirectInitValue', + 'Grad', 'GradMin', 'OFDepRef']} + module_name = 'SurfaceRouting' def __init__(self, surface_routing_variable): self.var = surface_routing_variable diff --git a/src/lisflood/hydrological_modules/transmission.py b/src/lisflood/hydrological_modules/transmission.py index 7fb4a7ce..a3afcd80 100644 --- a/src/lisflood/hydrological_modules/transmission.py +++ b/src/lisflood/hydrological_modules/transmission.py @@ -21,15 +21,18 @@ from ..global_modules.settings import LisSettings, MaskInfo from ..global_modules.add1 import loadmap +from . import HydroModule -class transmission(object): +class transmission(HydroModule): """ # ************************************************************ # ***** Transmission loss ************************************ # ************************************************************ """ + input_files_keys = {'TransLoss': ['TransArea', 'TransSub', 'UpAreaTrans', 'TransPower1']} + module_name = 'Transmission' def __init__(self, transmission_variable): self.var = transmission_variable diff --git a/src/lisflood/hydrological_modules/waterabstraction.py b/src/lisflood/hydrological_modules/waterabstraction.py index e43cf70a..3a9774b0 100644 --- a/src/lisflood/hydrological_modules/waterabstraction.py +++ b/src/lisflood/hydrological_modules/waterabstraction.py @@ -23,14 +23,27 @@ from ..global_modules.add1 import loadmap, decompress, compressArray, readnetcdf, readmapsparse from ..global_modules.settings import get_calendar_type, calendar_inconsistency_warning, LisSettings, MaskInfo +from . import HydroModule -class waterabstraction(object): +class waterabstraction(HydroModule): """ # ************************************************************ # ***** Water abstraction ************************************ # ************************************************************ """ + input_files_keys = {'wateruse': ['WUsePercRemain', 'maxNoWateruse', 'GroundwaterBodies', 'FractionGroundwaterUsed', + 'FractionNonConventionalWaterUsed', 'FractionLakeReservoirWaterUsed', + 'EFlowThreshold', 'WUseRegion', 'IrrigationMult', 'IndustryConsumptiveUseFraction', + 'WaterReUseFraction', 'EnergyConsumptiveUseFraction', + 'LivestockConsumptiveUseFraction', 'ConveyanceEfficiency', + 'LeakageFraction', 'LeakageReductionFraction', 'WaterSavingFraction', + 'DomesticConsumptiveUseFraction', 'LeakageWaterLoss', + 'DomesticDemandMaps', 'IndustrialDemandMaps', 'LivestockDemandMaps', + 'EnergyDemandMaps', 'IrrigationType', 'IrrigationEfficiency'], + 'groundwaterSmooth': ['LZSmoothRange'], + 'wateruseRegion': ['WUseRegion']} + module_name = 'WaterAbstraction' def __init__(self, waterabstraction_variable): self.var = waterabstraction_variable @@ -188,16 +201,10 @@ def initial(self): self.var.IrriLossCUM = maskinfo.in_zero() # Cumulative irrigation loss [mm] - # abstractionCUM = maskinfo.in_zero() # not used # Cumulative abstraction from surface water [mm] - # IrrigationWaterDemand = maskinfo.in_zero() - # if not(option['riceIrrigation']): - # self.var.TotalAbstractionFromSurfaceWaterM3 = maskinfo.in_zero() self.var.TotalAbstractionFromSurfaceWaterM3 = maskinfo.in_zero() self.var.TotalAbstractionFromGroundwaterM3 = maskinfo.in_zero() - # IrrigationAbstractionFromGroundwaterM3 = maskinfo.in_zero() - # LiveDomAbstractionFromGroundwaterM3 = maskinfo.in_zero() # not used self.var.TotalIrrigationAbstractionM3 = maskinfo.in_zero() self.var.TotalPaddyRiceIrrigationAbstractionM3 = maskinfo.in_zero() self.var.TotalLivestockAbstractionM3 = maskinfo.in_zero() @@ -207,13 +214,18 @@ def initial(self): self.var.ConveyanceEfficiency = loadmap('ConveyanceEfficiency') self.var.GroundwaterRegionPixels = np.take( - np.bincount(self.var.WUseRegionC, weights=self.var.GroundwaterBodies), self.var.WUseRegionC) + np.bincount(self.var.WUseRegionC, weights=self.var.GroundwaterBodies), + self.var.WUseRegionC + ) self.var.AllRegionPixels = np.take( - np.bincount(self.var.WUseRegionC, weights=self.var.GroundwaterBodies * 0.0 + 1.0), self.var.WUseRegionC) + np.bincount(self.var.WUseRegionC, weights=self.var.GroundwaterBodies * 0.0 + 1.0), + self.var.WUseRegionC + ) self.var.RatioGroundWaterUse = self.var.AllRegionPixels / (self.var.GroundwaterRegionPixels + 0.01) self.var.FractionGroundwaterUsed = np.minimum( self.var.FractionGroundwaterUsed * self.var.RatioGroundWaterUse, - 1 - self.var.FractionNonConventionalWaterUsed) + 1 - self.var.FractionNonConventionalWaterUsed + ) # FractionGroundwaterUsed is a percentage given at national scale # since the water needs to come from the GroundwaterBodies pixels, # the fraction needs correction for the non-Groundwaterbodies; this is done here @@ -257,10 +269,6 @@ def dynamic(self): averageyearflag=True) * self.var.DtDay else: # Read from stack of maps in NetCDF format. Get time step corresponding to model step. - # self.var.DomesticDemandMM = readnetcdfsparse(binding['DomesticDemandMaps'], self.var.currentTimeStep(), self.var.DomesticDemandMM) - # self.var.IndustrialDemandMM = readnetcdfsparse(binding['IndustrialDemandMaps'], self.var.currentTimeStep(), self.var.IndustrialDemandMM) - # self.var.LivestockDemandMM = readnetcdfsparse(binding['LivestockDemandMaps'], self.var.currentTimeStep(), self.var.LivestockDemandMM) - # self.var.EnergyDemandMM = readnetcdfsparse(binding['EnergyDemandMaps'], self.var.currentTimeStep(), self.var.EnergyDemandMM) # added management for sub-daily model time steps self.var.DomesticDemandMM = readnetcdf(binding['DomesticDemandMaps'], self.var.currentTimeStep(), @@ -309,8 +317,7 @@ def dynamic(self): self.var.DomesticAbstractionMM = self.var.DomesticDemandMM * self.var.DomesticWaterSavingConstant * self.var.DomesticLeakageConstant # Domestic Water Abstraction (mm per day), already taking into account water saving in households and leakage of the supply network # Domestic water abstraction is larger if there is leakage, but is smaller if there is water savings - self.var.LeakageMM = ( - self.var.DomesticLeakageConstant - 1) * self.var.DomesticDemandMM * self.var.DomesticWaterSavingConstant + self.var.LeakageMM = (self.var.DomesticLeakageConstant - 1) * self.var.DomesticDemandMM * self.var.DomesticWaterSavingConstant # Leakage in mm per day self.var.LeakageLossMM = self.var.LeakageMM * self.var.LeakageWaterLossFraction # The leakage amount that is lost (evaporated) diff --git a/src/lisflood/hydrological_modules/waterlevel.py b/src/lisflood/hydrological_modules/waterlevel.py index fab5e4c8..807f8d37 100644 --- a/src/lisflood/hydrological_modules/waterlevel.py +++ b/src/lisflood/hydrological_modules/waterlevel.py @@ -20,14 +20,17 @@ from ..global_modules.add1 import loadmap from ..global_modules.settings import LisSettings +from . import HydroModule -class waterlevel(object): +class waterlevel(HydroModule): """ # ************************************************************ # ***** WATER LEVEL ***************************************** # ************************************************************ """ + input_files_keys = {'simulateWaterLevels': ['FloodPlainWidth']} + module_name = 'WaterLevel' def __init__(self, waterlevel_variable): self.var = waterlevel_variable diff --git a/src/lisflood/lisf1grid.py b/src/lisflood/lisf1grid.py deleted file mode 100644 index b176bd12..00000000 --- a/src/lisflood/lisf1grid.py +++ /dev/null @@ -1,209 +0,0 @@ -# """ -# -# Copyright 2019 European Union -# -# Licensed under the EUPL, Version 1.2 or as soon they will be approved by the European Commission subsequent versions of the EUPL (the "Licence"); -# -# You may not use this work except in compliance with the Licence. -# You may obtain a copy of the Licence at: -# -# https://joinup.ec.europa.eu/sites/default/files/inline-files/EUPL%20v1_2%20EN(1).txt -# -# Unless required by applicable law or agreed to in writing, software distributed under the Licence is distributed on an "AS IS" basis, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the Licence for the specific language governing permissions and limitations under the Licence. -# -# ###################################################################### -# -# ## #### ###### ######## ## ####### ####### ######## -# ## ## ## ## ## ## ## ## ## ## ## ## -# ## ## ## ## ## ## ## ## ## ## ## -# ## ## ###### ###### ## ## ## ## ## ## ## -# ## ## ## ## ## ## ## ## ## ## ## -# ## ## ## ## ## ## ## ## ## ## ## ## -# ######## #### ###### ## ######## ####### ####### ######## -# -# ###################################################################### -# """ -# from __future__ import print_function, absolute_import -# -# from lisflood.global_modules.utils import LisfloodRunInfo -# from lisflood.global_modules.settings import check_simulation_dates -# -# __authors__ = "Ad de Roo, Peter Burek, Johan van der Knijff, Niko Wanders" -# __version__ = "Version: 2.02" -# __date__ = "23 Apr 2016" -# __copyright__ = "Copyright 2016, European Commission - Joint Research Centre" -# __maintainer__ = "Ad de Roo" -# __status__ = "Operation" -# -# -# # to work with the new grid engine JRC - workaround with error on pyexpat -# -# from .Lisflood_initial import * -# from .Lisflood_dynamic import * -# from .Lisflood_monteCarlo import * -# from .Lisflood_EnKF import * -# -# -# class LisfloodModel(LisfloodModel_ini, LisfloodModel_dyn, LisfloodModel_monteCarlo, LisfloodModel_EnKF): -# """ Joining the initial and the dynamic part -# of the Lisflood model -# """ -# -# # ================================================== -# # ============== LISFLOOD execute ================== -# # ================================================== -# -# def Lisfloodexe(): -# -# optionBinding(settings, optionxml) -# # read all the possible option for modelling and for generating output -# # read the settingsfile with all information about the catchments(s) -# # and the choosen option for mdelling and output -# bindkey = sorted(binding.keys()) -# -# # this prevent from using relative path in settings! -# -# check_simulation_dates('StepStart', 'StepEnd') -# -# if option['InitLisflood']: print("INITIALISATION RUN") -# print("Start - End: ",modelSteps[0]," - ", modelSteps[1]) -# if Flags['loud']: -# print("%-6s %10s %11s\n" %("Step","Date","Discharge")) -# -# Lisflood = LisfloodModel() -# stLisflood = DynamicFramework(Lisflood, firstTimestep=modelSteps[0], lastTimeStep=modelSteps[1]) -# stLisflood.rquiet = True -# stLisflood.rtrace = False -# -# -# """ -# ---------------------------------------------- -# Monte Carlo and Ensemble Kalman Filter setting -# ---------------------------------------------- -# """ -# -# try: -# EnKFset = option['EnKF'] -# except: -# EnKFset = 0 -# try: -# MCset = option['MonteCarlo'] -# except: -# MCset = 0 -# if option['InitLisflood']: -# MCset = 0 -# EnKFset = 0 -# if EnKFset and not MCset: -# msg = "Trying to run EnKF with only 1 ensemble member \n" -# raise LisfloodError(msg) -# if EnKFset and FilterSteps[0] == 0: -# msg = "Trying to run EnKF without filter timestep specified \nRunning LISFLOOD in Monte Carlo mode \n" -# print(LisfloodWarning(msg)) -# EnKFset = 0 -# if MCset and EnsMembers[0] <= 1: -# msg = "Trying to run Monte Carlo simulation with only 1 member \nRunning LISFLOOD in deterministic mode \n" -# print(LisfloodWarning(msg)) -# MCset = 0 -# if MCset: -# mcLisflood = MonteCarloFramework(stLisflood, nrSamples=EnsMembers[0]) -# if nrCores[0] > 1: -# mcLisflood.setForkSamples(True, nrCPUs=nrCores[0]) -# if EnKFset: -# kfLisflood = EnsKalmanFilterFramework(mcLisflood) -# kfLisflood.setFilterTimesteps(FilterSteps) -# print(LisfloodRunInfo(mode = "Ensemble Kalman Filter", outputDir = outputDir[0], Steps = len(FilterSteps), ensMembers=EnsMembers[0], Cores=nrCores[0])) -# kfLisflood.run() -# else: -# print(LisfloodRunInfo(mode = "Monte Carlo", outputDir = outputDir[0], ensMembers=EnsMembers[0], Cores=nrCores[0])) -# mcLisflood.run() -# else: -# """ -# ---------------------------------------------- -# Deterministic run -# ---------------------------------------------- -# """ -# print(LisfloodRunInfo(mode = "Deterministic", outputDir = outputDir[0])) -# stLisflood.run() -# # cProfile.run('stLisflood.run()') -# # python -m cProfile -o l1.pstats lisf1.py settingsNew3.xml -# # gprof2dot -f pstats l1.pstats | dot -Tpng -o callgraph.png -# -# if Flags['printtime']: -# print("\n\nTime profiling") -# print("%2s %-17s %10s %8s" %("No","Name","time[s]","%")) -# div = 1 -# timeSum = np.array(timeMesSum) -# if MCset: -# div = div * EnsMembers[0] -# if EnKFset: -# div = div * (len(FilterSteps)+1) -# if EnKFset or MCset: -# timePrint = np.zeros(len(timeSum)/div) -# for i in range(len(timePrint)): -# timePrint[i] = np.sum(timeSum[range(i, len(timeSum), len(timeSum)/div)]) -# else: -# timePrint = timeSum -# for i in range(len(timePrint)): -# print("%2i %-17s %10.2f %8.1f" %(i,timeMesString[i],timePrint[i],100 * timePrint[i] / timePrint[-1])) -# i=1 # ?!? -# -# # ================================================== -# # ============== USAGE ============================== -# # ================================================== -# -# -# def usage(): -# """ prints some lines describing how to use this program -# which arguments and parameters it accepts, etc -# """ -# print('LisfloodPy - Lisflood using pcraster Python framework') -# print('Authors: ', __authors__) -# print('Version: ', __version__) -# print('Date: ', __date__) -# print('Status: ', __status__) -# print(""" -# Arguments list: -# settings.xml settings file -# -# -q --quiet output progression given as . -# -v --veryquiet no output progression is given -# -l --loud output progression given as time step, date and discharge -# -c --check input maps and stack maps are checked, output for each input map BUT no model run -# -h --noheader .tss file have no header and start immediately with the time series -# -t --printtime the computation time for hydrological modules are printed -# """) -# sys.exit(1) -# -# def headerinfo(): -# -# print("LisfloodPy ",__version__," ",__date__) -# print(""" -# Water balance and flood simulation model for large catchments\n -# (C) Institute for Environment and Sustainability -# Joint Research Centre of the European Commission -# TP122, I-21020 Ispra (Va), Italy\n""") -# -# -# # ================================================== -# # ============== MAIN ============================== -# # ================================================== -# -# if __name__ == "__main__": -# -# if len(sys.argv) < 2: -# usage() -# -# -# LF_Path = os.path.dirname(sys.argv[0]) -# LF_Path = os.path.abspath(LF_Path) -# optionxml = os.path.normpath(LF_Path + "/OptionTserieMaps.xml") -# -# settings = sys.argv[1] # setting.xml file -# -# args = sys.argv[2:] -# globalFlags(args) -# # setting of global flag e.g checking input maps, producing more output information -# if not(Flags['veryquiet']) and not(Flags['quiet']) : headerinfo() -# Lisfloodexe() diff --git a/src/lisflood/main.py b/src/lisflood/main.py index 57c03ec2..e23c5be2 100644 --- a/src/lisflood/main.py +++ b/src/lisflood/main.py @@ -28,6 +28,7 @@ from pcraster.framework import EnsKalmanFilterFramework, MonteCarloFramework +from .global_modules.checkers import ModulesInputs from .global_modules.zusatz import DynamicFramework from .Lisflood_EnKF import LisfloodModel_EnKF from .Lisflood_dynamic import LisfloodModel_dyn @@ -53,16 +54,20 @@ class LisfloodModel(LisfloodModel_ini, LisfloodModel_dyn, LisfloodModel_monteCar def lisfloodexe(lissettings=None): - # read options and bindings and launch Lisflood model computation - # returns option binding and ReportSteps - global dictionaries - - # optionBinding(settings, optionxml) - # lissettings = LisSettings(settings) _ = CDFFlags(uuid.uuid4()) if isinstance(lissettings, str): + # we passed file path lissettings = LisSettings(lissettings) else: + # deal with LisSettings instance lissettings = LisSettings.instance() if lissettings is None else lissettings + + try: + ModulesInputs.check(lissettings) + except LisfloodError as e: + print(e) + sys.exit(1) + binding = lissettings.binding option = lissettings.options report_steps = lissettings.report_steps @@ -225,5 +230,4 @@ def main(): flags = lissettings.flags if not (flags['veryquiet'] or flags['quiet']): headerinfo() - lisfloodexe(lissettings) diff --git a/src/settings_tpl.xml b/src/settings_tpl.xml index 4f699248..70a4020f 100644 --- a/src/settings_tpl.xml +++ b/src/settings_tpl.xml @@ -31,7 +31,6 @@ You can use builtin path variables in this template and reference to other paths -