Python update (PR #170 , formerly PR #166)

* close #84 * support Python 3.6 – 3.9 (in addition to Python 2.7) * use six and other compatibility hacks - replaced outdated merge_dicts function in config.py with recursive method that works in python 2 - added `__deepcopy__` method in fep.py with method that works in python 3. - use `__future__` imports - use logger.warning() everywhere * Updated tests - avoid reading errors when opening pickle file generated in Python 2 - replaced outdated yield tests - fixed tempdir imports (replace with pytest fixtures later) * updated mdpow scripts Python3 compatibility (untested, see #172 ) * Updated ci workflow to test supported Python versions * updated AUTHORS * updated CHANGES Co-authored-by: Oliver Beckstein <[email protected]>
Becksteinlab · Jul 30, 2021 · 4d2498b · 4d2498b
1 parent d279f50
commit 4d2498b
Show file tree

Hide file tree

Showing 36 changed files with 274 additions and 145 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -22,20 +22,29 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest, macOS-latest]
-        python-version: [2.7] 
-        #python-version: [2.7, 3.8] 
-        #python-version: [2.7, 3.6, 3.7, 3.8, 3.9]
+        # only test all GROMACS version on the oldest and latest
+        # Python to keep the testing matrix manageable and only use 2
+        # macos runners (latest GROMACS, oldes and latest Python)
+
+        os: [ubuntu-latest]
+        python-version: [2.7, 3.9]
         gromacs-version: ["4.6.5", "2018.6", "2020.6", "2021.1"]
-        # only test one GROMACS version on macOS to keep the testing
-        # matrix manageable
-        exclude:
-          - os: macOS-latest
-            gromacs-version: "4.6.5"
-          - os: macOS-latest
-            gromacs-version: "2018.6"
-          - os: macOS-latest
-            gromacs-version: "2020.6"
+        include:
+          - os: ubuntu-latest
+            python-version: 3.6
+            gromacs-version: 2021.1
+          - os: ubuntu-latest
+            python-version: 3.7
+            gromacs-version: 2021.1
+          - os: ubuntu-latest
+            python-version: 3.8
+            gromacs-version: 2021.1
+          - os: macos-latest
+            python-version: 2.7
+            gromacs-version: 2021.1
+          - os: macos-latest
+            python-version: 3.9
+            gromacs-version: 2021.1
 
     env:
       MPLBACKEND: agg

diff --git a/AUTHORS b/AUTHORS
@@ -32,4 +32,4 @@ their first commit. GitHub handle is optional.
 2021
 ----
 
-- Alia Lescoulie (ALescoul)
+- Alia Lescoulie (ALescoulie)
diff --git a/CHANGES b/CHANGES
@@ -6,11 +6,13 @@ Add summary of changes for each release. Use ISO dates. Reference
 GitHub issues numbers and PR numbers.
 
 2021-07-xx    0.7.0
-orbeckst, VOD555, ALescoul
+orbeckst, VOD555, ALescoulie
 
 Changes
 
 * renamed package to MDPOW
+* support Python 3.7 -- 3.9 on Linux and macOS (#84)
+* tested with GROMACS 4.6.5, 2018.6, 2020.6, 2021.1 (PR #159, #164)
 * removed all generated docs from package
 * config parser MERGES user runinput.yml with the package defaults
   (#8)
@@ -50,7 +52,7 @@ Enhancements
 * supported CHARMM and AMBER forcefield, including PRM parameter files
   with the "setup.prm" parameter in the configuration file (#104)
 * supported wet-octanol mixed solvent boxtype but this only works with
-  GROMACS >= 2018 (#98)  
+  GROMACS >= 2018 (#98)
 * support OPLS-AA TIP4PD, AMBER TIP4PEW and cyclohexane, and CHARMM TIP5P
   and cyclohexane solvent types (#141)
 * supported forcefield options in scripts (#123)
@@ -72,7 +74,7 @@ Fixes
   found. Either supply the name of the right checkpoint file or do not use
   -append": mdpow.run.runMD_or_exit() does not anymore add -append to GROMACS
   invocation (#128)
-* fixed mdpow-pow and mdpow-pcw scripts (#138)  
+* fixed mdpow-pow and mdpow-pcw scripts (#138)
 * fixed template em_charmm.mdp to use standard CHARMM non-bonded parameters
   for energy minimization (PR #155)
 

diff --git a/INSTALL.rst b/INSTALL.rst
@@ -2,10 +2,9 @@
  Quick installation instructions for *MDPOW*
 =============================================
 
-**Only Python 2.7 is supported** (Python 3 is *not* supported, see
-`#84 <https://github.com/Becksteinlab/MDPOW/issues/84>`_).  Python 2.7
-is rock-stable and frozen but not officially supported anymore by the
-Python developers.
+MDPOW is compatible with Python 2.7 and 3.7+ and tested
+on Ubuntu and Mac OS. Python 2.7 is rock-stable and frozen but
+not officially supported anymore by the Python developers.
 
 We recommend that you install MDPOW in a virtual environment.
 
@@ -30,14 +29,24 @@ GROMACS_.
 Conda environment with pre-requisites
 -------------------------------------
 
-We make a conda environment with the latest packages for Python 2.7
+To make a conda environment with the latest packages for Python 2.7
 and name it *mdpow*; this installs the larger dependencies that are
 pre-requisites for MDPOW::
 
- conda create -c conda-forge -n mdpow python=2.7 numpy scipy 'matplotlib<3.3' 'mdanalysis<2' 'mdanalysistests<2' pyyaml
+ conda create -c conda-forge -n mdpow python=2.7 numpy scipy 'matplotlib<3.3' 'mdanalysis<2' 'mdanalysistests<2' pyyaml six
  conda activate mdpow  
  pip install gromacswrapper
 
+
+For Python 3.7 and up.
+
+*Note:* with Pandas version 1.3 there is an error with `Alchemlyb <https://github.com/alchemistry/alchemlyb>`_
+(see `issue #147 <https://github.com/alchemistry/alchemlyb/issues/147>`_) which will be fixed in Alchemlyb 0.5.::
+
+ conda create -c conda-forge -n mdpow python=3.7 numpy scipy 'matplotlib' 'mdanalysis' 'mdanalysistests' pyyaml six
+ conda activate mdpow
+ pip install gromacswrapper
+
 Installation from source
 ------------------------
 

diff --git a/README.rst b/README.rst
@@ -10,7 +10,7 @@
 *MDPOW* is a python package that automates the calculation of
 solvation free energies via molecular dynamics (MD) simulations. In
 particular, it facilitates the computation of partition
-coeffcients. Currently implemented:
+coefficients. Currently implemented:
 
 - *water-octanol* partition coefficient (|P_ow|)
 - *water-cyclohexane* partition coefficient (|P_cw|)
@@ -43,8 +43,7 @@ Documentation
 Installation
 ------------
 
-See `INSTALL`_ for detailed instructions. Note that
-**only Python 2.7** is supported.
+See `INSTALL`_ for detailed instructions. MDPOW currently supports Python 2.7 and Python 3.7 to 3.9.
 
 You will also need `Gromacs`_ (currently tested with versions 4.6.5,
 2018, 2020, 2021 but 2016 and 2019 should also work).

diff --git a/doc/sphinx/source/index.txt b/doc/sphinx/source/index.txt
@@ -43,9 +43,9 @@ software package [#GromacsWrapperFramework]_. MDPOW is tested with
 * Gromacs 4.6.5
 * Gromacs 2018.6
 * Gromacs 2020.6
-* Gromacs 2021.1 
+* Gromacs 2021.1
 
-but versions 5.x, 2016.x, and 2019.x should also work. 
+but versions 5.x, 2016.x, and 2019.x should also work.
 It should be possible to use any of these Gromacs versions without
 further adjustments, thanks to the underlying GromacsWrapper library
 [#GromacsWrapperFramework]_.
@@ -99,9 +99,9 @@ also change (rarely) between MINOR releases. *MINOR* releases can
 introduce new functionality or deprecate old ones.
 
 The version information can be accessed from the attribute
-:data:`gromacs.__version__`.
+:data:`mdpow.__version__`.
 
-.. autodata:: gromacs.__version__
+.. autodata:: mdpow.__version__
 
 .. _semantic versioning: https://semver.org	      
 
@@ -126,7 +126,7 @@ For current issues and open feature requests please look through the
 
 
 
-.. Hide to use with alabaster
+.. Hide to use with RTD theme
 .. Contents:
 
 .. toctree::

diff --git a/mdpow/config.py b/mdpow/config.py
@@ -90,18 +90,20 @@
 
 """
 
-from __future__ import absolute_import
+from __future__ import absolute_import, division
+
+import six
 
 import os, errno
 from pkg_resources import resource_filename, resource_listdir
 import yaml
 
 import numpy as np
+import gromacs.utilities
 
 import logging
 logger = logging.getLogger("mdpow.config")
 
-
 # Reading of configuration files
 # ------------------------------
 
@@ -112,15 +114,15 @@
 
 def merge_dicts(user, default):
     """Merge two dictionaries recursively.
-
-    Based on https://stackoverflow.com/a/823240/334357
+    Uses recursive method to accurately
+    merge nested dictionaries
     """
-    if isinstance(user, dict) and isinstance(default, dict):
-        for k, v in default.iteritems():
-            if k not in user:
-                user[k] = v
-            else:
-                user[k] = merge_dicts(user[k], v)
+    for key in default:
+        if key in user:
+            if isinstance(user[key], dict) and isinstance(default[key], dict):
+                merge_dicts(user[key], default[key])
+        else:
+            user[key] = default[key]
     return user
 
 
@@ -167,8 +169,12 @@ def get(self, section, option):
            Prior versions would convert case-insensitively (e.g. "NONE"
            and "none")
         """
-        value = self.conf[section][option]
-        return value if value != "None" else None
+
+        try:
+            value = self.conf[section][option]
+            return value if value != "None" else None
+        except TypeError:
+            return None
 
     # TODO:
     # The YAML parser does automatic conversion: the following
@@ -233,7 +239,7 @@ def get_configuration(filename=None):
 def modify_gromacs_environment(name, value):
     from gromacs.environment import flags
     if flags[name] != value:
-        logger.warn("Changing GromacsWrapper environment: flags[%(name)r] = %(value)r", vars())
+        logger.warning("Changing GromacsWrapper environment: flags[%(name)r] = %(value)r", vars())
         flags[name] = value
 
 def set_gromacsoutput(cfg):
@@ -285,7 +291,8 @@ def get_template(t):
     :Raises:    :exc:`ValueError` if no file can be located.
 
     """
-    templates = [_get_template(s) for s in asiterable(t)]
+    # Not sure if this is the best way to get asiterables
+    templates = [_get_template(s) for s in gromacs.utilities.asiterable(t)]
     if len(templates) == 1:
          return templates[0]
     return templates
@@ -308,7 +315,7 @@ def get_templates(t):
     :Raises:    :exc:`ValueError` if no file can be located.
 
     """
-    return [_get_template(s) for s in utilities.asiterable(t)]
+    return [_get_template(s) for s in gromacs.utilities.asiterable(t)]
 
 def _get_template(t):
     """Return a single template *t*."""
@@ -339,8 +346,8 @@ def _get_template(t):
 
 def iterable(obj):
     """Returns ``True`` if *obj* can be iterated over and is *not* a  string."""
-    if isinstance(obj, basestring):
-        return False    # avoid iterating over characters of a string
+    if isinstance(obj, six.string_types):
+        return False  # avoid iterating over characters of a string
 
     if hasattr(obj, 'next'):
         return True    # any iterator will do
@@ -416,6 +423,6 @@ def asiterable(obj):
     logger.info("Using the bundled force fields from GMXLIB=%(includedir)r.", vars())
     logger.info("If required, override this behaviour by setting the environment variable GMXLIB yourself.")
 else:
-    logger.warn("Using user-supplied environment variable GMXLIB=%r to find force fields", os.environ['GMXLIB'])
+    logger.warning("Using user-supplied environment variable GMXLIB=%r to find force fields", os.environ['GMXLIB'])
     logger.info("(You can use the MDPOW default by executing 'unset GMXLIB' in your shell before running MDPOW.)")
 
diff --git a/mdpow/equil.py b/mdpow/equil.py
@@ -29,11 +29,13 @@
 .. autodata:: DIST
 """
 
-from __future__ import absolute_import, with_statement
+from __future__ import absolute_import, division
+
+from six.moves import cPickle as pickle
 
 import os, errno
 import shutil
-import cPickle
+
 import MDAnalysis as mda
 
 try:
@@ -227,7 +229,7 @@ def save(self, filename=None):
         else:
             self.filename = filename
         with open(filename, 'wb') as f:
-            cPickle.dump(self, f, protocol=cPickle.HIGHEST_PROTOCOL)
+            pickle.dump(self, f, protocol=pickle.HIGHEST_PROTOCOL)
         logger.debug("Instance pickled to %(filename)r" % vars())
 
     def load(self, filename=None):
@@ -238,7 +240,7 @@ def load(self, filename=None):
                 logger.warning("No filename known, trying name %r", self.filename)
             filename = self.filename
         with open(filename, 'rb') as f:
-            instance = cPickle.load(f)
+            instance = pickle.load(f)
         self.__dict__.update(instance.__dict__)
         logger.debug("Instance loaded from %(filename)r" % vars())
 
@@ -273,8 +275,8 @@ def assinglet(m):
                 self.mdp[key] = fn.replace(basedir, prefix)
             except AttributeError:
                 pass
-        logger.warn("make_paths_relative(): check/manually adjust %s.dirs.includes = %r !",
-                    self.__class__.__name__, self.dirs.includes)
+        logger.warning("make_paths_relative(): check/manually adjust %s.dirs.includes = %r !",
+                       self.__class__.__name__, self.dirs.includes)
 
     def topology(self, itp='drug.itp', prm=None, **kwargs):
         """Generate a topology for compound *molecule*.
@@ -305,7 +307,7 @@ def topology(self, itp='drug.itp', prm=None, **kwargs):
         itp = os.path.realpath(itp)
         _itp = os.path.basename(itp)
 
-        if prm==None:
+        if prm is None:
             prm_kw = ''
         else:
             prm = os.path.realpath(prm)
@@ -466,8 +468,8 @@ def _MD(self, protocol, **kwargs):
         # so instead of fuffing with GMXLIB we just dump it into the directory
         try:
             shutil.copy(config.topfiles['residuetypes.dat'], self.dirs[protocol])
-        except:
-            logger.warn("Failed to copy 'residuetypes.dat': mdrun will likely fail to write a final structure")
+        except IOError:
+            logger.warning("Failed to copy 'residuetypes.dat': mdrun will likely fail to write a final structure")
 
         self.journal.completed(protocol)
         return params