From 1dd636fa489ef4b214739a81fe4ffc22b554f561 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Thu, 22 Apr 2021 12:28:56 -0700 Subject: [PATCH 01/14] Help out MSVS 2017 with derived class swap method Visual Studio 2017 does not like our implementation of the swap() method for our custom set subclass (MSVS 2015 and gcc have no issues). Resolve by explicitly calling the base class method. --- include/RMF/internal/SharedDataData.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/RMF/internal/SharedDataData.h b/include/RMF/internal/SharedDataData.h index 9586e380..0b6ee6cd 100644 --- a/include/RMF/internal/SharedDataData.h +++ b/include/RMF/internal/SharedDataData.h @@ -44,7 +44,7 @@ struct TypeData : RMF_SMALL_UNORDERED_MAP, KeyData > { P::operator=(o); return *this; } - void swap(TypeData& o) { std::swap

(*this, o); } + void swap(TypeData& o) { P::swap(o); } }; #define RMF_SHARED_DATA_TYPE_PARENT(Traits, UCName) , public TypeData From aed6e60de2f7af9879d2208b966c5d905c3f78e0 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Fri, 23 Apr 2021 13:24:29 -0700 Subject: [PATCH 02/14] Link to conda packages at conda-forge --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index d30f81ef..4afbaa79 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # RMF # [![Build Status](https://github.com/salilab/rmf/workflows/build/badge.svg?branch=develop)](https://github.com/salilab/rmf/actions?query=workflow%3Abuild) +[![conda package](https://img.shields.io/conda/vn/conda-forge/rmf.svg)](https://anaconda.org/conda-forge/rmf) [![codecov](https://codecov.io/gh/salilab/rmf/branch/develop/graph/badge.svg)](https://codecov.io/gh/salilab/rmf) [![Code Climate](https://codeclimate.com/github/salilab/rmf/badges/gpa.svg)](https://codeclimate.com/github/salilab/rmf) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4701672.svg)](https://doi.org/10.5281/zenodo.4701672) From 8fddad66276627736ebdbe250a8934c3dc325c96 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Fri, 23 Apr 2021 16:46:09 -0700 Subject: [PATCH 03/14] Add basic install instructions --- doc/Installation.md | 71 +++++++++++++++++++++++++++++++++++++++++++++ doc/Main.md | 1 + 2 files changed, 72 insertions(+) create mode 100644 doc/Installation.md diff --git a/doc/Installation.md b/doc/Installation.md new file mode 100644 index 00000000..af8d0bf3 --- /dev/null +++ b/doc/Installation.md @@ -0,0 +1,71 @@ +Installation {#installation} +============ + +# Binary installation # {#binary} + +The simplest way to obtain RMF is by using a pre-built binary, either +standalone or as part of [IMP](https://integrativemodeling.org). + +Standalone: if you are using [Anaconda Python](https://www.anaconda.com/), install with + +``` +conda install -c conda-forge rmf +``` + +IMP: Download an IMP binary (which includes RMF) from the +[IMP download page](https://integrativemodeling.org/download.html). + +# Source code installation # {#source} + +## Prerequisites {#installation_prereqs} + +In order to build from source, you will need: + +- [CMake](https://cmake.org) (2.8.12 or later; 3.14 or later is recommended) +- [Boost](https://www.boost.org) (1.53 or later; Boost.Iostreams must be built + with its [zlib filter enabled](https://www.boost.org/doc/libs/1_67_0/libs/iostreams/doc/installation.html)) +- [Python](https://www.python.org) (2.7 or later, or any version of Python 3) +- [SWIG](http://www.swig.org) (1.3.40 or later; 2.0.4 or later is needed + if you want to use Python 3) + +If you want to be able to read older format RMF files, you will also need: + +- [HDF5](https://support.hdfgroup.org/HDF5/) (1.8 or later; 1.10 or 1.12 + should also work) + +## Download {#installation_download} + +- Use [git](https://git-scm.com/) to get the code + directly from our [GitHub repository](https://github.com/salilab/rmf) + with something like: + + git clone -b main https://github.com/salilab/rmf.git + + (the `main` branch tracks the most recent stable + release; alternatively you can use `develop` to get the most recent code). + +## Compilation {#installation_compilation} + +Make a separate directory to keep the compiled version of RMF in (it's tidier +to keep this separate from the source code, and if you need to later you can +just delete this directory without affecting the source). Set up the build +with CMake, then finally compile it, with something like: + + mkdir build + cd build + cmake -DCMAKE_BUILD_TYPE=Release + make -j8 + +## Testing {#installation_testing} +Once the compilation is complete, you can optionally run the test suite. +Test are run using `ctest`. A good start is to run `ctest --output-on-failure`. + +Tests are labeled with the cost of the test, so to run just the cheap tests, +use `ctest -L CHEAP`. + +## Installation {#installation_install} + +Once everything is compiled (and optionally tested) you can install RMF +by simply running `make install`. If you opted to install in a non-standard +location, it is up to you to set up your environment variables so that RMF +can be found (you may need to set `PATH`, `PYTHONPATH`, and `LD_LIBRARY_PATH`). diff --git a/doc/Main.md b/doc/Main.md index 73481509..27ff74dd 100644 --- a/doc/Main.md +++ b/doc/Main.md @@ -10,6 +10,7 @@ It can be built standalone, although it is generally used as part of [IMP](http://integrativemodeling.org). See +- [Installation](\ref installation) for information on obtaining and installing RMF. - [RMF viewers](\ref viewing) for more information about viewing RMF files, - [RMF file format](\ref format) for more information about the files, - [RMF Library](\ref library) for more information about using the library. From 3895a7074a4a02ab378bcdf19ce310d5e589507b Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Thu, 6 May 2021 15:33:02 -0700 Subject: [PATCH 04/14] Fix block quote --- doc/Installation.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/Installation.md b/doc/Installation.md index af8d0bf3..09c0bb62 100644 --- a/doc/Installation.md +++ b/doc/Installation.md @@ -8,9 +8,7 @@ standalone or as part of [IMP](https://integrativemodeling.org). Standalone: if you are using [Anaconda Python](https://www.anaconda.com/), install with -``` -conda install -c conda-forge rmf -``` + conda install -c conda-forge rmf IMP: Download an IMP binary (which includes RMF) from the [IMP download page](https://integrativemodeling.org/download.html). From 1ec7b301422e48555e1f7093602d704899cde688 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Mon, 24 May 2021 12:13:36 -0700 Subject: [PATCH 05/14] Squashed 'cmake_modules/' changes from 6fb1fe51..c80a68d4 c80a68d4 Use clang not Darwin version to detect c++14 support 94539adb Add a simple module to find an RMF installation git-subtree-dir: cmake_modules git-subtree-split: c80a68d422a11e87ddc9f2a65fe7e1f7aa4e61cc --- cmake_modules/FindRMF.cmake | 44 ++++++++++++++++++++++++++++++++ cmake_modules/IMPFindC++11.cmake | 10 ++++++-- 2 files changed, 52 insertions(+), 2 deletions(-) create mode 100644 cmake_modules/FindRMF.cmake diff --git a/cmake_modules/FindRMF.cmake b/cmake_modules/FindRMF.cmake new file mode 100644 index 00000000..2c87be14 --- /dev/null +++ b/cmake_modules/FindRMF.cmake @@ -0,0 +1,44 @@ +#[=======================================================================[.rst: +FindRMF +------- + +Try to find RMF + +Result Variables +^^^^^^^^^^^^^^^^ + +This module defines the following variables: + +``RMF_FOUND`` + system has RMF +``RMF_INCLUDE_PATH`` + the RMF include directory +``RMF_SWIG_PATH`` + the directory containing SWIG (.i) files for RMF +``RMF_LIBRARY`` + Link this to use RMF +``RMF_VERSION_STRING`` + the version of RMF found + + +#]=======================================================================] + + +find_path(RMF_INCLUDE_PATH RMF.h PATH_SUFFIXES include) +find_path(RMF_SWIG_PATH RMF.i PATH_SUFFIXES share/RMF/swig) +if (NOT RMF_LIBRARY) + find_library(RMF_LIBRARY NAMES RMF PATH_SUFFIXES lib) +endif() + +if (RMF_INCLUDE_PATH AND EXISTS "${RMF_INCLUDE_PATH}/RMF/config.h") + file(STRINGS "${RMF_INCLUDE_PATH}/RMF/config.h" RMF_MAJOR_H REGEX "#define RMF_VERSION_MAJOR +([0-9]+)") + file(STRINGS "${RMF_INCLUDE_PATH}/RMF/config.h" RMF_MINOR_H REGEX "#define RMF_VERSION_MINOR +([0-9]+)") + string(REGEX REPLACE " *#define RMF_VERSION_MAJOR +([0-9]+) *" "\\1" RMF_VERSION_MAJOR "${RMF_MAJOR_H}") + string(REGEX REPLACE " *#define RMF_VERSION_MINOR +([0-9]+) *" "\\1" RMF_VERSION_MINOR "${RMF_MINOR_H}") + set(RMF_VERSION_STRING "${RMF_VERSION_MAJOR}.${RMF_VERSION_MINOR}") +endif() + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(RMF + REQUIRED_VARS RMF_LIBRARY RMF_INCLUDE_PATH RMF_SWIG_PATH + VERSION_VAR RMF_VERSION_STRING) diff --git a/cmake_modules/IMPFindC++11.cmake b/cmake_modules/IMPFindC++11.cmake index a60e4197..bf55e3d2 100644 --- a/cmake_modules/IMPFindC++11.cmake +++ b/cmake_modules/IMPFindC++11.cmake @@ -29,8 +29,14 @@ if(IMP_CXX11) set(IMP_CXX11_FLAGS "--std=c++0x" CACHE INTERNAL "" FORCE) endif() elseif("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang") - # Modern clang uses C++14 by default; don't force older C++11 - if(APPLE AND DARWIN_VERSION GREATER 18) + execute_process(COMMAND ${CMAKE_CXX_COMPILER} --version + OUTPUT_VARIABLE CLANG_VERSION) + if(CLANG_VERSION MATCHES "clang version ([0-9.]+)") + set(CLANG_VERSION ${CMAKE_MATCH_1}) + endif() + message(STATUS "clang version: ${CLANG_VERSION}") + # Modern clang (6 or later) uses C++14 by default; don't force older C++11 + if(CLANG_VERSION VERSION_GREATER 6.0) message(STATUS "Using clang C++11 (or later) support") # c++11's std::move (which boost/CGAL use) doesn't work until # OS X 10.9 (Darwin version 13) From a2dfdc2d358e37746d36879203fcec377f0c3266 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Mon, 14 Jun 2021 17:30:47 -0700 Subject: [PATCH 06/14] Switch to Codecov Action --- .github/workflows/build.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index afc9d992..a18d4236 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -67,8 +67,8 @@ jobs: make -j 2 export LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so ctest -j 2 --output-on-failure -L ${{ matrix.tests }} - - name: Upload coverage to codecov + - name: Combine coverage run: | - cd build - (cd coverage && coverage combine && mv .coverage ..) - bash <(curl -s https://codecov.io/bash) + cd build/coverage + coverage combine && mv .coverage ../.. + - uses: codecov/codecov-action@v1 From 0f300492b18393ff8e9da4c5d7bdaee9173bbd42 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Mon, 14 Jun 2021 17:31:07 -0700 Subject: [PATCH 07/14] Ignore Mac junk --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index afc1f64b..448722b1 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,7 @@ # exclude patterns (uncomment them if you want to use them): # *.[oa] # *~ +.DS_Store *.os *.o *~ From 666f9a5da51e621755fdcbe4ed88aa286c17bfa5 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Mon, 14 Jun 2021 17:53:30 -0700 Subject: [PATCH 08/14] Use C++17 mode with newer clang --- cmake_modules/IMPFindC++11.cmake | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cmake_modules/IMPFindC++11.cmake b/cmake_modules/IMPFindC++11.cmake index bf55e3d2..509bec1a 100644 --- a/cmake_modules/IMPFindC++11.cmake +++ b/cmake_modules/IMPFindC++11.cmake @@ -37,7 +37,8 @@ if(IMP_CXX11) message(STATUS "clang version: ${CLANG_VERSION}") # Modern clang (6 or later) uses C++14 by default; don't force older C++11 if(CLANG_VERSION VERSION_GREATER 6.0) - message(STATUS "Using clang C++11 (or later) support") + message(STATUS "Enabling clang C++17 support") + set(IMP_CXX11_FLAGS "--std=c++17" CACHE INTERNAL "" FORCE) # c++11's std::move (which boost/CGAL use) doesn't work until # OS X 10.9 (Darwin version 13) elseif(APPLE AND DARWIN_VERSION LESS 13) From a9b95e1fcae70c499858bf119645215628d764bb Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Tue, 27 Jul 2021 11:40:57 -0700 Subject: [PATCH 09/14] Use canonical URLs Replace http: with https: URLs where appropriate and use canonical domain names. --- doc/Developers.md | 4 ++-- doc/FileFormat.md | 4 ++-- doc/Main.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/Developers.md b/doc/Developers.md index d2e91038..4313531f 100644 --- a/doc/Developers.md +++ b/doc/Developers.md @@ -9,11 +9,11 @@ This page documents various internal aspects of how the RMF library works and is # Developer tools # {#dev_tools} -RMF uses the [Salilab developer tools](http://www.github.com/salilab/developer_tools) to provide basic developer tools in a way that is shared with [IMP](http://integrativemodeling.org). This is included as a subrepository. Read the docs of that repository for more information. +RMF uses the [Salilab developer tools](https://github.com/salilab/developer_tools) to provide basic developer tools in a way that is shared with [IMP](https://integrativemodeling.org). This is included as a subrepository. Read the docs of that repository for more information. # CMake tools # {#cmake_tools} -RMF uses [Salilab cmake modules](http://www.github.com/salilab/cmake_modules) to provide some extensions to [CMake](http://cmake.org). This is included as a subrepository. +RMF uses [Salilab cmake modules](https://github.com/salilab/cmake_modules) to provide some extensions to [CMake](https://cmake.org). This is included as a subrepository. # Decorators # {#makedecorators} diff --git a/doc/FileFormat.md b/doc/FileFormat.md index 666938c3..c1983f95 100644 --- a/doc/FileFormat.md +++ b/doc/FileFormat.md @@ -169,7 +169,7 @@ nodes. When adding data to an %RMF file that is just to be used for internal consumption, one should create a new category. For example, -[IMP](http://www.integrativemodeling.org) defines an ''imp'' category +[IMP](https://integrativemodeling.org) defines an ''imp'' category when arbitrary particle data are stored. If, instead, the data is likely to be a general interest, it probably @@ -183,7 +183,7 @@ methods are supported: files with suffix `.rmf` and `.rmfz` and buffers in memory. The current format stores the structure in an [Avro Object -Container](http://avro.apache.org/docs/1.7.5/spec.html#Object+Container+Files). If +Container](https://avro.apache.org/docs/1.7.5/spec.html#Object+Container+Files). If the `.rmfz` suffix is used, the contents are compressed. The structure is stored as a series of records, each containing either a frame or static data (there can be multiple static data frames - they are diff --git a/doc/Main.md b/doc/Main.md index 27ff74dd..ebdf721e 100644 --- a/doc/Main.md +++ b/doc/Main.md @@ -7,7 +7,7 @@ and score data. The RMF source code is available [on github](https://github.com/salilab/rmf/). It can be built standalone, although it is generally used as part of -[IMP](http://integrativemodeling.org). +[IMP](https://integrativemodeling.org). See - [Installation](\ref installation) for information on obtaining and installing RMF. @@ -25,4 +25,4 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 From 9ad4661cb48ed19cc968bb55ac84cff86c8d08a4 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Mon, 9 Aug 2021 11:44:17 -0700 Subject: [PATCH 10/14] Work around boost::flat_map segfault with gcc 11 We previously observed a segfault with Fedora 32 and gcc 10 in boost::flat_map::operator[]. The same issue seems to occur with both Fedora 34 and conda-build, both of which use gcc 11, so use the same workaround there. --- include/RMF/internal/SharedDataData.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/RMF/internal/SharedDataData.h b/include/RMF/internal/SharedDataData.h index 0b6ee6cd..456e2b62 100644 --- a/include/RMF/internal/SharedDataData.h +++ b/include/RMF/internal/SharedDataData.h @@ -28,9 +28,9 @@ struct KeyData : public RMF_LARGE_UNORDERED_MAP { KeyData() {} }; -// boost flat_map::operator[] segfaults on Fedora 32; use std::map instead +// boost flat_map::operator[] segfaults on Fedora 32+; use std::map instead template -#if __GNUC__ == 10 && __GNUC_MINOR__ == 0 +#if __GNUC__ >= 10 struct TypeData : std::map, KeyData > { typedef std::map, KeyData > P; #else From f3e2a095a40a135ae8bed15633525c36f5733337 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Wed, 18 Aug 2021 16:46:07 -0700 Subject: [PATCH 11/14] Warn about missing RMF_OVERRIDE If we override a virtual method in a subclass, we should use RMF_OVERRIDE so that the compiler reports an error if we got the signature wrong (such that it hides the base method instead of overriding it). Add -Wsuggest-override to gcc warnings (if gcc is new enough) to point out places where we missed this. --- include/RMF/compiler_macros.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/include/RMF/compiler_macros.h b/include/RMF/compiler_macros.h index ac45c9d6..64ba32ad 100644 --- a/include/RMF/compiler_macros.h +++ b/include/RMF/compiler_macros.h @@ -159,6 +159,18 @@ #define RMF_GCC_PROTOTYPES #endif +// Warn about missing RMF_OVERRIDE on virtual methods if gcc is new enough +#if __GNUC__ > 5 || (__GNUC__ == 5 && __GNUC_MINOR__ >= 1) +#ifdef RMF_SWIG_WRAPPER +#define RMF_GCC_OVERRIDE +#else +#define RMF_GCC_OVERRIDE \ + RMF_GCC_PRAGMA(diagnostic warning "-Wsuggest-override") +#endif +#else +#define RMF_GCC_OVERRIDE +#endif + #define RMF_COMPILER_WARNINGS \ RMF_GCC_PRAGMA(diagnostic warning "-Wall") \ RMF_GCC_PRAGMA(diagnostic warning "-Wextra") \ @@ -166,8 +178,8 @@ RMF_GCC_PRAGMA(diagnostic warning "-Wcast-align") \ RMF_GCC_PRAGMA(diagnostic warning "-Woverloaded-virtual") \ RMF_GCC_PRAGMA(diagnostic ignored "-Wconversion") \ - RMF_GCC_PRAGMA(diagnostic warning \ - "-Wundef") RMF_GCC_PROTOTYPES RMF_GCC_CXX0X_COMPAT + RMF_GCC_PRAGMA(diagnostic warning "-Wundef") \ + RMF_GCC_PROTOTYPES RMF_GCC_CXX0X_COMPAT RMF_GCC_OVERRIDE #elif defined(_MSC_VER) #define RMF_COMPILER_WARNINGS \ From f18213a65cee5112b4ef7b730093286605499ca0 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Wed, 18 Aug 2021 16:46:37 -0700 Subject: [PATCH 12/14] Add missing RMF_OVERRIDE annotation --- include/RMF/exceptions.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/RMF/exceptions.h b/include/RMF/exceptions.h index 2586eac3..62231460 100644 --- a/include/RMF/exceptions.h +++ b/include/RMF/exceptions.h @@ -31,7 +31,7 @@ class RMFEXPORT Exception : public virtual std::exception, public: RMF_CXX11_DEFAULT_COPY_CONSTRUCTOR(Exception); Exception(); - const char* what() const RMF_NOEXCEPT; + const char* what() const RMF_NOEXCEPT RMF_OVERRIDE; virtual ~Exception() RMF_NOEXCEPT; }; From 4895bff9d22381882ac38180bdd025e22bdc7c00 Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Tue, 12 Oct 2021 12:56:18 -0700 Subject: [PATCH 13/14] Squashed 'tools/dev_tools/' changes from e5d19974..8e5ec80a 8e5ec80a Don't use distutils in modern Python git-subtree-dir: tools/dev_tools git-subtree-split: 8e5ec80a597a80e6474fe564fae56b9c272abcc0 --- tools/dev_tools/cleanup_code.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tools/dev_tools/cleanup_code.py b/tools/dev_tools/cleanup_code.py index 4d506b53..dd664d47 100755 --- a/tools/dev_tools/cleanup_code.py +++ b/tools/dev_tools/cleanup_code.py @@ -14,7 +14,10 @@ except ImportError: from Queue import Queue # python2 from threading import Thread -import distutils.spawn +try: + from shutil import which # python3.3 or later +except ImportError: + from distutils.spawn import find_executable as which sys.path.append(os.path.split(sys.argv[0])) import python_tools @@ -60,13 +63,13 @@ if options.clang_format == "auto": options.clang_format = None for name in ["clang-format-3.4", "clang-format"]: - if distutils.spawn.find_executable(name): + if which(name): options.clang_format = name break if options.autopep8 == "auto": options.autopep8 = None for name in ["autopep8"]: - if distutils.spawn.find_executable(name): + if which(name): options.autopep8 = name break From 1f9442a766022801db5fc356acbcdd78843d5daf Mon Sep 17 00:00:00 2001 From: Ben Webb Date: Wed, 1 Dec 2021 11:37:08 -0800 Subject: [PATCH 14/14] Add changes for 1.3.1 release --- ChangeLog.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/ChangeLog.md b/ChangeLog.md index 55695fde..5c23cc54 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,6 +1,12 @@ Change Log {#changelog} ========== +# 1.3.1 - 2021-12-01 # {#changelog_1_3_1} +- Various build fixes for newer versions of gcc and clang. +- Build fixes for Fedora 33 or later. +- Add support for Python 3.10. +- Minor documentation improvements. + # 1.3 - 2021-04-21 # {#changelog_1_3} - All RMF binaries now support the --version flag, and the RMF library itself is versioned.