base_demand, demand_pattern and demand_category at junctions go to/fr…

…om gis (#447)

* Add ability to input single demand from giw/dict

* Add test

* update doctests to account for column name changes

* change default return in junction properties to reflect default values in add_junction

* fix doctests

* move demand attributes from optional to base. fix docstest accordingly

* fix doctest

* fix doctest by adding sort to column name list

* rewrite portions of IO documentation for clarity

* add cross references for methods

---------

Co-authored-by: Angus <[email protected]>
Co-authored-by: kbonney <[email protected]>

documentation/gis.rst

@@ -119,13 +119,13 @@ For example, the junctions GeoDataFrame contains the following information:
     :skipif: gpd is None
 
     >>> print(wn_gis.junctions.head())
-          elevation  initial_quality                   geometry
-    name                                                       
-    10      216.408        5.000e-04  POINT (20.00000 70.00000)
-    11      216.408        5.000e-04  POINT (30.00000 70.00000)
-    12      213.360        5.000e-04  POINT (50.00000 70.00000)
-    13      211.836        5.000e-04  POINT (70.00000 70.00000)
-    21      213.360        5.000e-04  POINT (30.00000 40.00000)
+          base_demand demand_pattern  elevation  initial_quality demand_category                   geometry
+    name                                                                                                   
+    10          0.000              1    216.408        5.000e-04            None  POINT (20.00000 70.00000)
+    11          0.009              1    216.408        5.000e-04            None  POINT (30.00000 70.00000)
+    12          0.009              1    213.360        5.000e-04            None  POINT (50.00000 70.00000)
+    13          0.006              1    211.836        5.000e-04            None  POINT (70.00000 70.00000)
+    21          0.009              1    213.360        5.000e-04            None  POINT (30.00000 40.00000)
 
 Each GeoDataFrame contains attributes and geometry:
 
@@ -341,23 +341,23 @@ and then translates the GeoDataFrames coordinates to EPSG:3857.
 
     >>> wn_gis = wntr.network.to_gis(wn, crs='EPSG:4326')
     >>> print(wn_gis.junctions.head())
-          elevation  initial_quality                   geometry
-    name                                                       
-    10      216.408        5.000e-04  POINT (20.00000 70.00000)
-    11      216.408        5.000e-04  POINT (30.00000 70.00000)
-    12      213.360        5.000e-04  POINT (50.00000 70.00000)
-    13      211.836        5.000e-04  POINT (70.00000 70.00000)
-    21      213.360        5.000e-04  POINT (30.00000 40.00000)
+          base_demand demand_pattern  elevation  initial_quality demand_category                   geometry
+    name                                                                                                   
+    10          0.000              1    216.408        5.000e-04            None  POINT (20.00000 70.00000)
+    11          0.009              1    216.408        5.000e-04            None  POINT (30.00000 70.00000)
+    12          0.009              1    213.360        5.000e-04            None  POINT (50.00000 70.00000)
+    13          0.006              1    211.836        5.000e-04            None  POINT (70.00000 70.00000)
+    21          0.009              1    213.360        5.000e-04            None  POINT (30.00000 40.00000)
 
     >>> wn_gis.to_crs('EPSG:3857')
     >>> print(wn_gis.junctions.head())
-          elevation  initial_quality                          geometry
-    name                                                              
-    10      216.408        5.000e-04  POINT (2226389.816 11068715.659)
-    11      216.408        5.000e-04  POINT (3339584.724 11068715.659)
-    12      213.360        5.000e-04  POINT (5565974.540 11068715.659)
-    13      211.836        5.000e-04  POINT (7792364.356 11068715.659)
-    21      213.360        5.000e-04   POINT (3339584.724 4865942.280)
+          base_demand demand_pattern  elevation  initial_quality demand_category                          geometry
+    name                                                                                                          
+    10          0.000              1    216.408        5.000e-04            None  POINT (2226389.816 11068715.659)
+    11          0.009              1    216.408        5.000e-04            None  POINT (3339584.724 11068715.659)
+    12          0.009              1    213.360        5.000e-04            None  POINT (5565974.540 11068715.659)
+    13          0.006              1    211.836        5.000e-04            None  POINT (7792364.356 11068715.659)
+    21          0.009              1    213.360        5.000e-04            None   POINT (3339584.724 4865942.280)
 
 Snap point geometries to the nearest point or line
 ----------------------------------------------------

documentation/model_io.rst

@@ -206,29 +206,28 @@ GeoJSON files
 GeoJSON files are commonly used to store geographic data structures. 
 More information on GeoJSON files can be found at https://geojson.org.
 
-When reading GeoJSON files into WNTR, only a set of valid column names can be used.
-Valid GeoJSON column names can be obtained using the 
-:class:`~wntr.network.io.valid_gis_names` function. By default, the function
-returns all column names, both required and optional.
-The following example returns valid GeoJSON column names for junctions.
+When reading GeoJSON files into WNTR, the file should contain columns from the set of valid column names. 
+Valid GeoJSON column names can be obtained using the :class:`~wntr.network.io.valid_gis_names` function. 
+By default, the function returns a complete set of required and optional column names. 
+A minimal list of column names containing commonly used attributes can be obtained by setting ``complete_list`` to False. 
+The minimal set correspond with attributes used in :class:`~wntr.network.model.WaterNetworkModel.add_junction`, :class:`~wntr.network.model.WaterNetworkModel.add_tank`, etc.  
+Columns that are optional (i.e., ``initial_quality``) and not included in the GeoJSON file are defined using default values.
+
+The following examples return the complete and minimal lists of valid GeoJSON column names for junctions.
 
 .. doctest::
     :skipif: gpd is None
 
     >>> geojson_column_names = wntr.network.io.valid_gis_names()
     >>> print(geojson_column_names['junctions'])
-    ['name', 'elevation', 'geometry', 'emitter_coefficient', 'initial_quality', 'minimum_pressure', 'required_pressure', 'pressure_exponent', 'tag']
-
-A minimal list of required column names can also be obtained by setting ``complete_list`` to False.
-Column names that are optional (i.e., ``initial_quality``) and not included in the GeoJSON file are 
-defined using default values.
+    ['name', 'base_demand', 'demand_pattern', 'elevation', 'demand_category', 'geometry', 'emitter_coefficient', 'initial_quality', 'minimum_pressure', 'required_pressure', 'pressure_exponent', 'tag']
 
 .. doctest::
     :skipif: gpd is None
 
     >>> geojson_column_names = wntr.network.io.valid_gis_names(complete_list=False)
     >>> print(geojson_column_names['junctions'])
-    ['name', 'elevation', 'geometry']
+    ['name', 'base_demand', 'demand_pattern', 'elevation', 'demand_category', 'geometry']
 
 Note that GeoJSON files can contain additional custom column names that are assigned to WaterNetworkModel objects.
 
@@ -301,31 +300,32 @@ To use Esri Shapefiles in WNTR, several formatting requirements are enforced:
   node and link attribute names are often longer.  For this reason, it is
   assumed that the first 10 characters of each attribute are unique.  
 
-* To create WaterNetworkModel from Shapefiles, a set of valid field names are required.
+* When reading Shapefiles files into WNTR, the file should contain fields from the set of valid column names.
   Valid Shapefiles field names can be obtained using the 
   :class:`~wntr.network.io.valid_gis_names` function. By default, the function
-  returns all column names, both required and optional.
+  returns a complete set of required and optional field names. 
+  A minimal list of field names containing commonly used attributes can be obtained by setting ``complete_list`` to False. 
+  The minimal set correspond with attributes used in `add_junction`, `add_tank`, etc.  
+  Fields that are optional (i.e., ``initial_quality``) and not included in the Shapefile are defined using default values.
+
   For Shapefiles, the `truncate_names` input parameter should be set to 10 (characters).
-  The following example returns valid Shapefile field names for junctions.
+  The following examples return the complete and minimal lists of valid Shapefile field names for junctions.
   Note that attributes like ``minimum_pressure`` are truncated to ``minimum_pr``. 
 
   .. doctest::
       :skipif: gpd is None
 
       >>> shapefile_field_names = wntr.network.io.valid_gis_names(truncate_names=10)
       >>> print(shapefile_field_names['junctions'])
-      ['name', 'elevation', 'geometry', 'emitter_co', 'initial_qu', 'minimum_pr', 'required_p', 'pressure_e', 'tag']
-
-  A minimal list of required field names can also be obtained by setting ``complete_list`` to False.
-  Field names that are optional (i.e., ``initial_quality``) and not included in the Shapefile are defined using default values.
+      ['name', 'base_deman', 'demand_pat', 'elevation', 'demand_cat', 'geometry', 'emitter_co', 'initial_qu', 'minimum_pr', 'required_p', 'pressure_e', 'tag']
 
   .. doctest::
       :skipif: gpd is None
 
       >>> shapefile_field_names = wntr.network.io.valid_gis_names(complete_list=False, 
       ...    truncate_names=10)
       >>> print(shapefile_field_names['junctions'])
-      ['name', 'elevation', 'geometry']
+      ['name', 'base_deman', 'demand_pat', 'elevation', 'demand_cat', 'geometry']
 
 * Shapefiles can contain additional custom field names that are assigned to WaterNetworkModel objects.
 

wntr/gis/network.py

@@ -143,6 +143,7 @@ def _extract_geodataframe(df, crs=None, valid_base_names=None,
 
                 # Add back in valid base attributes that had all None values
                 cols = list(set(valid_base_names) - set(df.columns))
+                cols.sort()
                 if len(cols) > 0:
                     df[cols] = None
 

wntr/network/base.py

@@ -257,7 +257,7 @@ def to_dict(self):
         d['node_type'] = self.node_type
         for k in dir(self):
             if not k.startswith('_') and \
-              k not in ['demand', 'base_demand', 'head', 'leak_area', 'leak_demand',
+              k not in ['demand', 'head', 'leak_area', 'leak_demand',
                         'leak_discharge_coeff', 'leak_status', 'level', 'pressure', 'quality', 'vol_curve', 'head_timeseries']:
                 try:
                     val = getattr(self, k)

wntr/network/elements.py

@@ -79,8 +79,11 @@ class Junction(Node):
     # base and optional attributes used to create a Junction in _from_dict
     # base attributes are used in add_junction
     _base_attributes = ["name", 
+                        "base_demand",
+                        "demand_pattern",
                         "elevation", 
-                        "coordinates"]
+                        "coordinates",
+                        "demand_category"]
     _optional_attributes = ["emitter_coefficient",
                             "initial_quality", 
                             "minimum_pressure", 
@@ -216,6 +219,32 @@ def base_demand(self):
     def base_demand(self, value):
         raise RuntimeWarning('The base_demand property is read-only. Please modify using demand_timeseries_list[0].base_value.')
 
+    @property
+    def demand_pattern(self):
+        """Get the pattern_name of the first demand in the demand_timeseries_list.
+
+        This is a read-only property.
+        """
+        if len(self.demand_timeseries_list) > 0:
+            return self.demand_timeseries_list[0].pattern_name
+        return None
+    @demand_pattern.setter
+    def demand_pattern(self, value):
+        raise RuntimeWarning('The demand_pattern property is read-only. Please modify using demand_timeseries_list[0].pattern_name')
+
+    @property
+    def demand_category(self):
+        """Get the category of the first demand in the demand_timeseries_list.
+
+        This is a read-only property.
+        """
+        if len(self.demand_timeseries_list) > 0:
+            return self.demand_timeseries_list[0].category
+        return None
+    @demand_category.setter
+    def demand_category(self, value):
+        raise RuntimeWarning('The demand_category property is read-only. Please modify using demand_timeseries_list[0].category.')
+
     def add_leak(self, wn, area, discharge_coeff=0.75, start_time=None, end_time=None):
         """
         Add a leak control to the water network model

wntr/network/io.py

@@ -117,18 +117,18 @@ def from_dict(d: dict, append=None):
                 if dl is not None and len(dl) > 0:
                     base_demand = dl[0].setdefault("base_val", 0.0)
                     pattern_name = dl[0].setdefault("pattern_name")
-                    category = dl[0].setdefault("category")
+                    demand_category = dl[0].setdefault("category")
                 else:
-                    base_demand = 0.0
-                    pattern_name = None
-                    category = None
+                    base_demand = node.setdefault('base_demand',0.0)
+                    pattern_name = node.setdefault('pattern_name')
+                    demand_category = node.setdefault('demand_category')
                 wn.add_junction(
                     name=name,
                     base_demand=base_demand,
                     demand_pattern=pattern_name,
                     elevation=node.setdefault("elevation"),
                     coordinates=node.setdefault("coordinates", list()),
-                    demand_category=category,
+                    demand_category=demand_category,
                 )
                 j = wn.get_node(name)
                 j.emitter_coefficient = node.setdefault("emitter_coefficient")

wntr/tests/test_gis.py

@@ -138,6 +138,9 @@ def test_wn_to_gis(self):
         assert set(['start_node_name', 'end_node_name', 'geometry']).issubset(self.gis_data.pipes.columns)
         assert set(['start_node_name', 'end_node_name', 'geometry']).issubset(self.gis_data.pumps.columns)
         #assert set(['start_node_name', 'end_node_name', 'geometry']).issubset(self.gis_data.valves.columns) # Net1 has no valves
+
+        #check base_demand and demand_pattern attrivutes
+        assert set(['base_demand','demand_pattern']).issubset(self.gis_data.junctions.columns)
 
     def test_gis_to_wn(self):