diff --git a/doc/.template.coding.ipynb b/doc/.template.coding.ipynb index 6ad935a..569b80b 100644 --- a/doc/.template.coding.ipynb +++ b/doc/.template.coding.ipynb @@ -477,11 +477,95 @@ "See [the latest GitHub Actions results](https://github.com/salilab/imp_coding_tutorial/actions)\n", "and the [the latest code coverage reports](https://codecov.io/gh/salilab/imp_coding_tutorial)." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# RMF support {#rmf}\n", + "\n", + "We can extend on our basic class by adding support for RMF (for ease of comparison, we'll do this in a new class `MyRestraint2` that is a copy of `MyRestraint`.)\n", + "\n", + "When IMP writes restraints to RMF files, only basic information is included - namely\n", + " * *static* information (that does not change during the course of a simulation): the name of the restraint, and the particles it acts on; and\n", + " * *dynamic* information (which generally changes from frame to frame): the total score of the restraint.\n", + "\n", + "We can add extra static or dynamic information to the RMF file, by overriding the ``~IMP.Restraint.get_static_info`` or ``~IMP.Restraint.get_dynamic_info`` methods, respectively. Each returns an ``IMP::RestraintInfo`` object which is a simple list of key:value pairs. Here we'll add the force constant (which is static information) to the RMF file by declaring the method in the [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n", + "\n", + "```cpp\n", + "RestraintInfo *get_static_info() const override;\n", + "```\n", + "\n", + "Next, we provide an implementation of the method in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`:\n", + "\n", + "```cpp\n", + "RestraintInfo *MyRestraint2::get_static_info() const {\n", + " IMP_NEW(RestraintInfo, ri, ());\n", + " ri->add_string(\"type\", \"IMP.foo.MyRestraint2\");\n", + " ri->add_float(\"force constant\", k_);\n", + " return ri.release();\n", + "}\n", + "```\n", + "\n", + "> The convention in IMP is that if static restraint information is\n", + "> provided, it should include a \"type\" string which gives the full\n", + "> name of the restraint. All key names should be lower case words\n", + "> separated by spaces, e.g. \"force constant\" rather than \"ForceConstant\".\n", + "\n", + "Of course, the new method can and should be tested - see the `test_static_info` method in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py), `test/test_restraint2.py`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Serialization support {#serial}\n", + "\n", + "We can further extend our `MyRestraint2` class by adding support for [serialization](@ref serialization).\n", + "When an IMP object is serialized its internal state - such as the values of its member variables - is\n", + "written to or read in from a file, string or stream. This allows for individual objects or an entire IMP run to be saved and later restored, or to be sent from one machine to another. IMP uses the [cereal library](https://uscilab.github.io/cereal/) to implement serialization in C++. In Python, the objects can be loaded or saved using the ``pickle`` module.\n", + "\n", + "To add basic serialization support to our class, we first must add the cereal headers to our\n", + "[C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n", + "\n", + "```cpp\n", + "#include \n", + "```\n", + "\n", + "Then we add a default constructor and a new private method `serialize` to the same [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), which the cereal library will use to both read and write the class state:\n", + "\n", + "```cpp\n", + " MyRestraint2() {}\n", + "private:\n", + " friend class cereal::access;\n", + " template void serialize(Archive &ar) {\n", + " ar(cereal::base_class(this), p_, k_);\n", + " }\n", + " IMP_OBJECT_SERIALIZE_DECL(MyRestraint2);\n", + "```\n", + "\n", + "The default constructor (constructor which takes no arguments) is used to create an empty `MyRestraint2` object when deserializing - first the empty object is constructed, and then the class state is filled in from the serialization data.\n", + "\n", + "The class state comprises the state of the base `Restraint` class (which is handled by `cereal::base_class`), plus the particle our restraint acts on (`p_`) and the force constant (`k_`). (The `friend` declaration used here allows the cereal library to call our `serialize` method, which normally it would not be able to do since the method is marked `private`.)\n", + "\n", + "The `IMP_OBJECT_SERIALIZE_DECL` macro is used to handle polymorphic classes, which includes most IMP restraints. It needs to be paired with a similar macro in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`, which uses the fully qualified name of the class:\n", + "```cpp\n", + "IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2);\n", + "```\n", + "\n", + "To add support for Python ``pickle``, we replace the `IMP_SWIG_OBJECT` macro in the [SWIG interface file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/pyext/swig.i-in), `pyext/swig.i-in`, with `IMP_SWIG_OBJECT_SERIALIZE`:\n", + "\n", + "```cpp\n", + "IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s);\n", + "```\n", + "\n", + "Serialization support should also be tested - see the `test_serialize` and `test_serialize_polymorphic` methods in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py)." + ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -495,7 +579,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.1" + "version": "3.11.4" } }, "nbformat": 4, diff --git a/doc/coding.ipynb b/doc/coding.ipynb index 960c310..3a5b042 100644 --- a/doc/coding.ipynb +++ b/doc/coding.ipynb @@ -21,6 +21,8 @@ " - [Source control](#sourcecontrol)\n", " - [Build and test](#buildtest)\n", " - [Automatic testing](#autotest)\n", + " - [RMF support](#rmf)\n", + " - [Serialization support](#serial)\n", "\n", "# Introduction\n", "\n", @@ -488,11 +490,95 @@ "See [the latest GitHub Actions results](https://github.com/salilab/imp_coding_tutorial/actions)\n", "and the [the latest code coverage reports](https://codecov.io/gh/salilab/imp_coding_tutorial)." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# RMF support\n", + "\n", + "We can extend on our basic class by adding support for RMF (for ease of comparison, we'll do this in a new class `MyRestraint2` that is a copy of `MyRestraint`.)\n", + "\n", + "When IMP writes restraints to RMF files, only basic information is included - namely\n", + " * *static* information (that does not change during the course of a simulation): the name of the restraint, and the particles it acts on; and\n", + " * *dynamic* information (which generally changes from frame to frame): the total score of the restraint.\n", + "\n", + "We can add extra static or dynamic information to the RMF file, by overriding the [get_static_info](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1Restraint.html#af07a7d34ad419288411fd39b97cfd2b2) or [get_dynamic_info](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1Restraint.html#a38bbb37b0e8dc92785b1834233f577c6) methods, respectively. Each returns an [IMP::RestraintInfo](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1RestraintInfo.html) object which is a simple list of key:value pairs. Here we'll add the force constant (which is static information) to the RMF file by declaring the method in the [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n", + "\n", + "```cpp\n", + "RestraintInfo *get_static_info() const override;\n", + "```\n", + "\n", + "Next, we provide an implementation of the method in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`:\n", + "\n", + "```cpp\n", + "RestraintInfo *MyRestraint2::get_static_info() const {\n", + " IMP_NEW(RestraintInfo, ri, ());\n", + " ri->add_string(\"type\", \"IMP.foo.MyRestraint2\");\n", + " ri->add_float(\"force constant\", k_);\n", + " return ri.release();\n", + "}\n", + "```\n", + "\n", + "> The convention in IMP is that if static restraint information is\n", + "> provided, it should include a \"type\" string which gives the full\n", + "> name of the restraint. All key names should be lower case words\n", + "> separated by spaces, e.g. \"force constant\" rather than \"ForceConstant\".\n", + "\n", + "Of course, the new method can and should be tested - see the `test_static_info` method in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py), `test/test_restraint2.py`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Serialization support\n", + "\n", + "We can further extend our `MyRestraint2` class by adding support for [serialization](https://integrativemodeling.org/2.19.0/doc/manual/serialization.html).\n", + "When an IMP object is serialized its internal state - such as the values of its member variables - is\n", + "written to or read in from a file, string or stream. This allows for individual objects or an entire IMP run to be saved and later restored, or to be sent from one machine to another. IMP uses the [cereal library](https://uscilab.github.io/cereal/) to implement serialization in C++. In Python, the objects can be loaded or saved using the [pickle](https://docs.python.org/3/library/pickle.html#module-pickle) module.\n", + "\n", + "To add basic serialization support to our class, we first must add the cereal headers to our\n", + "[C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n", + "\n", + "```cpp\n", + "#include \n", + "```\n", + "\n", + "Then we add a default constructor and a new private method `serialize` to the same [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), which the cereal library will use to both read and write the class state:\n", + "\n", + "```cpp\n", + " MyRestraint2() {}\n", + "private:\n", + " friend class cereal::access;\n", + " template void serialize(Archive &ar) {\n", + " ar(cereal::base_class(this), p_, k_);\n", + " }\n", + " IMP_OBJECT_SERIALIZE_DECL(MyRestraint2);\n", + "```\n", + "\n", + "The default constructor (constructor which takes no arguments) is used to create an empty `MyRestraint2` object when deserializing - first the empty object is constructed, and then the class state is filled in from the serialization data.\n", + "\n", + "The class state comprises the state of the base `Restraint` class (which is handled by `cereal::base_class`), plus the particle our restraint acts on (`p_`) and the force constant (`k_`). (The `friend` declaration used here allows the cereal library to call our `serialize` method, which normally it would not be able to do since the method is marked `private`.)\n", + "\n", + "The `IMP_OBJECT_SERIALIZE_DECL` macro is used to handle polymorphic classes, which includes most IMP restraints. It needs to be paired with a similar macro in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`, which uses the fully qualified name of the class:\n", + "```cpp\n", + "IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2);\n", + "```\n", + "\n", + "To add support for Python [pickle](https://docs.python.org/3/library/pickle.html#module-pickle), we replace the `IMP_SWIG_OBJECT` macro in the [SWIG interface file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/pyext/swig.i-in), `pyext/swig.i-in`, with `IMP_SWIG_OBJECT_SERIALIZE`:\n", + "\n", + "```cpp\n", + "IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s);\n", + "```\n", + "\n", + "Serialization support should also be tested - see the `test_serialize` and `test_serialize_polymorphic` methods in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py)." + ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -506,7 +592,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.1" + "version": "3.11.4" } }, "nbformat": 4, diff --git a/modules/foo/include/MyRestraint2.h b/modules/foo/include/MyRestraint2.h new file mode 100644 index 0000000..8a64cca --- /dev/null +++ b/modules/foo/include/MyRestraint2.h @@ -0,0 +1,36 @@ +#ifndef IMPFOO_MY_RESTRAINT2_H +#define IMPFOO_MY_RESTRAINT2_H + +#include +#include +#include + +IMPFOO_BEGIN_NAMESPACE + +class IMPFOOEXPORT MyRestraint2 : public Restraint { + ParticleIndex p_; + double k_; + +public: + MyRestraint2(Model *m, ParticleIndex p, double k); + void do_add_score_and_derivatives(ScoreAccumulator sa) const override; + ModelObjectsTemp do_get_inputs() const override; + IMP_OBJECT_METHODS(MyRestraint2); + + // RMF output support + RestraintInfo *get_static_info() const override; + + // Serialization support + MyRestraint2() {} +private: + friend class cereal::access; + template void serialize(Archive &ar) { + ar(cereal::base_class(this), p_, k_); + } + IMP_OBJECT_SERIALIZE_DECL(MyRestraint2); + +}; + +IMPFOO_END_NAMESPACE + +#endif /* IMPFOO_MY_RESTRAINT2_H */ diff --git a/modules/foo/pyext/swig.i-in b/modules/foo/pyext/swig.i-in index f56307d..a49549e 100644 --- a/modules/foo/pyext/swig.i-in +++ b/modules/foo/pyext/swig.i-in @@ -4,6 +4,9 @@ IMP_SWIG_OBJECT(IMP::foo, MyRestraint, MyRestraints); +IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s); + //%include "IMP/example/ExampleRestraint.h" %include "IMP/foo/MyRestraint.h" +%include "IMP/foo/MyRestraint2.h" diff --git a/modules/foo/src/MyRestraint2.cpp b/modules/foo/src/MyRestraint2.cpp new file mode 100644 index 0000000..5dca7ec --- /dev/null +++ b/modules/foo/src/MyRestraint2.cpp @@ -0,0 +1,34 @@ +#include +#include + +IMPFOO_BEGIN_NAMESPACE + +MyRestraint2::MyRestraint2(Model *m, ParticleIndex p, double k) + : Restraint(m, "MyRestraint%1%"), p_(p), k_(k) {} + +void MyRestraint2::do_add_score_and_derivatives(ScoreAccumulator sa) const { + core::XYZ d(get_model(), p_); + double score = .5 * k_ * square(d.get_z()); + if (sa.get_derivative_accumulator()) { + double deriv = k_ * d.get_z(); + d.add_to_derivative(2, deriv, *sa.get_derivative_accumulator()); + } + sa.add_score(score); +} + +ModelObjectsTemp MyRestraint2::do_get_inputs() const { + return ModelObjectsTemp(1, get_model()->get_particle(p_)); +} + +// RMF output support +RestraintInfo *MyRestraint2::get_static_info() const { + IMP_NEW(RestraintInfo, ri, ()); + ri->add_string("type", "IMP.foo.MyRestraint2"); + ri->add_float("force constant", k_); + return ri.release(); +} + +// Serialization support +IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2); + +IMPFOO_END_NAMESPACE diff --git a/modules/foo/test/test_restraint2.py b/modules/foo/test/test_restraint2.py new file mode 100644 index 0000000..077308a --- /dev/null +++ b/modules/foo/test/test_restraint2.py @@ -0,0 +1,64 @@ +from __future__ import print_function, division +import IMP +import IMP.test +import IMP.algebra +import IMP.core +import IMP.foo +import pickle + + +class Tests(IMP.test.TestCase): + + def test_my_restraint(self): + """Test scoring of MyRestraint2""" + m = IMP.Model() + p = m.add_particle("p") + d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3)) + r = IMP.foo.MyRestraint2(m, p, 10.) + self.assertAlmostEqual(r.evaluate(True), 45.0, delta=1e-4) + self.assertLess(IMP.algebra.get_distance(d.get_derivatives(), + IMP.algebra.Vector3D(0,0,30)), + 1e-4) + self.assertEqual(len(r.get_inputs()), 1) + + def test_static_info(self): + """Test static info of MyRestraint2""" + m = IMP.Model() + p = m.add_particle("p") + d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3)) + r = IMP.foo.MyRestraint2(m, p, 10.) + info = r.get_static_info() + self.assertEqual(info.get_number_of_string(), 1) + self.assertEqual(info.get_string_key(0), "type") + self.assertEqual(info.get_string_value(0), "IMP.foo.MyRestraint2") + + self.assertEqual(info.get_number_of_float(), 1) + self.assertEqual(info.get_float_key(0), "force constant") + self.assertAlmostEqual(info.get_float_value(0), 10.0, delta=0.001) + + def test_serialize(self): + """Test (un-)serialize of MyRestraint2""" + m = IMP.Model() + p = m.add_particle("p") + d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3)) + r = IMP.foo.MyRestraint2(m, p, 10.) + self.assertAlmostEqual(r.evaluate(False), 45.0, delta=1e-3) + dump = pickle.dumps(r) + newr = pickle.loads(dump) + self.assertAlmostEqual(newr.evaluate(False), 45.0, delta=1e-3) + + def test_serialize_polymorphic(self): + """Test (un-)serialize of MyRestraint2 via polymorphic pointer""" + m = IMP.Model() + p = m.add_particle("p") + d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3)) + r = IMP.foo.MyRestraint2(m, p, 10.) + sf = IMP.core.RestraintsScoringFunction([r]) + self.assertAlmostEqual(sf.evaluate(False), 45.0, delta=1e-3) + dump = pickle.dumps(sf) + newsf = pickle.loads(dump) + self.assertAlmostEqual(newsf.evaluate(False), 45.0, delta=1e-3) + + +if __name__ == '__main__': + IMP.test.main()