Skip to content

Commit

Permalink
Demonstrate RMF and serialization support
Browse files Browse the repository at this point in the history
  • Loading branch information
benmwebb committed Jun 23, 2023
1 parent 444a8b6 commit e3322bf
Show file tree
Hide file tree
Showing 6 changed files with 311 additions and 4 deletions.
88 changes: 86 additions & 2 deletions doc/.template.coding.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -477,11 +477,95 @@
"See [the latest GitHub Actions results](https://github.com/salilab/imp_coding_tutorial/actions)\n",
"and the [the latest code coverage reports](https://codecov.io/gh/salilab/imp_coding_tutorial)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# RMF support {#rmf}\n",
"\n",
"We can extend on our basic class by adding support for RMF (for ease of comparison, we'll do this in a new class `MyRestraint2` that is a copy of `MyRestraint`.)\n",
"\n",
"When IMP writes restraints to RMF files, only basic information is included - namely\n",
" * *static* information (that does not change during the course of a simulation): the name of the restraint, and the particles it acts on; and\n",
" * *dynamic* information (which generally changes from frame to frame): the total score of the restraint.\n",
"\n",
"We can add extra static or dynamic information to the RMF file, by overriding the ``~IMP.Restraint.get_static_info`` or ``~IMP.Restraint.get_dynamic_info`` methods, respectively. Each returns an ``IMP::RestraintInfo`` object which is a simple list of key:value pairs. Here we'll add the force constant (which is static information) to the RMF file by declaring the method in the [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n",
"\n",
"```cpp\n",
"RestraintInfo *get_static_info() const override;\n",
"```\n",
"\n",
"Next, we provide an implementation of the method in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`:\n",
"\n",
"```cpp\n",
"RestraintInfo *MyRestraint2::get_static_info() const {\n",
" IMP_NEW(RestraintInfo, ri, ());\n",
" ri->add_string(\"type\", \"IMP.foo.MyRestraint2\");\n",
" ri->add_float(\"force constant\", k_);\n",
" return ri.release();\n",
"}\n",
"```\n",
"\n",
"> The convention in IMP is that if static restraint information is\n",
"> provided, it should include a \"type\" string which gives the full\n",
"> name of the restraint. All key names should be lower case words\n",
"> separated by spaces, e.g. \"force constant\" rather than \"ForceConstant\".\n",
"\n",
"Of course, the new method can and should be tested - see the `test_static_info` method in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py), `test/test_restraint2.py`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Serialization support {#serial}\n",
"\n",
"We can further extend our `MyRestraint2` class by adding support for [serialization](@ref serialization).\n",
"When an IMP object is serialized its internal state - such as the values of its member variables - is\n",
"written to or read in from a file, string or stream. This allows for individual objects or an entire IMP run to be saved and later restored, or to be sent from one machine to another. IMP uses the [cereal library](https://uscilab.github.io/cereal/) to implement serialization in C++. In Python, the objects can be loaded or saved using the ``pickle`` module.\n",
"\n",
"To add basic serialization support to our class, we first must add the cereal headers to our\n",
"[C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n",
"\n",
"```cpp\n",
"#include <cereal/access.hpp>\n",
"```\n",
"\n",
"Then we add a default constructor and a new private method `serialize` to the same [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), which the cereal library will use to both read and write the class state:\n",
"\n",
"```cpp\n",
" MyRestraint2() {}\n",
"private:\n",
" friend class cereal::access;\n",
" template<class Archive> void serialize(Archive &ar) {\n",
" ar(cereal::base_class<Restraint>(this), p_, k_);\n",
" }\n",
" IMP_OBJECT_SERIALIZE_DECL(MyRestraint2);\n",
"```\n",
"\n",
"The default constructor (constructor which takes no arguments) is used to create an empty `MyRestraint2` object when deserializing - first the empty object is constructed, and then the class state is filled in from the serialization data.\n",
"\n",
"The class state comprises the state of the base `Restraint` class (which is handled by `cereal::base_class`), plus the particle our restraint acts on (`p_`) and the force constant (`k_`). (The `friend` declaration used here allows the cereal library to call our `serialize` method, which normally it would not be able to do since the method is marked `private`.)\n",
"\n",
"The `IMP_OBJECT_SERIALIZE_DECL` macro is used to handle polymorphic classes, which includes most IMP restraints. It needs to be paired with a similar macro in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`, which uses the fully qualified name of the class:\n",
"```cpp\n",
"IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2);\n",
"```\n",
"\n",
"To add support for Python ``pickle``, we replace the `IMP_SWIG_OBJECT` macro in the [SWIG interface file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/pyext/swig.i-in), `pyext/swig.i-in`, with `IMP_SWIG_OBJECT_SERIALIZE`:\n",
"\n",
"```cpp\n",
"IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s);\n",
"```\n",
"\n",
"Serialization support should also be tested - see the `test_serialize` and `test_serialize_polymorphic` methods in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -495,7 +579,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
"version": "3.11.4"
}
},
"nbformat": 4,
Expand Down
90 changes: 88 additions & 2 deletions doc/coding.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
" - [Source control](#sourcecontrol)\n",
" - [Build and test](#buildtest)\n",
" - [Automatic testing](#autotest)\n",
" - [RMF support](#rmf)\n",
" - [Serialization support](#serial)\n",
"\n",
"# Introduction<a id=\"introduction\"></a>\n",
"\n",
Expand Down Expand Up @@ -488,11 +490,95 @@
"See [the latest GitHub Actions results](https://github.com/salilab/imp_coding_tutorial/actions)\n",
"and the [the latest code coverage reports](https://codecov.io/gh/salilab/imp_coding_tutorial)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# RMF support<a id=\"rmf\"></a>\n",
"\n",
"We can extend on our basic class by adding support for RMF (for ease of comparison, we'll do this in a new class `MyRestraint2` that is a copy of `MyRestraint`.)\n",
"\n",
"When IMP writes restraints to RMF files, only basic information is included - namely\n",
" * *static* information (that does not change during the course of a simulation): the name of the restraint, and the particles it acts on; and\n",
" * *dynamic* information (which generally changes from frame to frame): the total score of the restraint.\n",
"\n",
"We can add extra static or dynamic information to the RMF file, by overriding the [get_static_info](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1Restraint.html#af07a7d34ad419288411fd39b97cfd2b2) or [get_dynamic_info](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1Restraint.html#a38bbb37b0e8dc92785b1834233f577c6) methods, respectively. Each returns an [IMP::RestraintInfo](https://integrativemodeling.org/2.19.0/doc/ref/classIMP_1_1RestraintInfo.html) object which is a simple list of key:value pairs. Here we'll add the force constant (which is static information) to the RMF file by declaring the method in the [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n",
"\n",
"```cpp\n",
"RestraintInfo *get_static_info() const override;\n",
"```\n",
"\n",
"Next, we provide an implementation of the method in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`:\n",
"\n",
"```cpp\n",
"RestraintInfo *MyRestraint2::get_static_info() const {\n",
" IMP_NEW(RestraintInfo, ri, ());\n",
" ri->add_string(\"type\", \"IMP.foo.MyRestraint2\");\n",
" ri->add_float(\"force constant\", k_);\n",
" return ri.release();\n",
"}\n",
"```\n",
"\n",
"> The convention in IMP is that if static restraint information is\n",
"> provided, it should include a \"type\" string which gives the full\n",
"> name of the restraint. All key names should be lower case words\n",
"> separated by spaces, e.g. \"force constant\" rather than \"ForceConstant\".\n",
"\n",
"Of course, the new method can and should be tested - see the `test_static_info` method in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py), `test/test_restraint2.py`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Serialization support<a id=\"serial\"></a>\n",
"\n",
"We can further extend our `MyRestraint2` class by adding support for [serialization](https://integrativemodeling.org/2.19.0/doc/manual/serialization.html).\n",
"When an IMP object is serialized its internal state - such as the values of its member variables - is\n",
"written to or read in from a file, string or stream. This allows for individual objects or an entire IMP run to be saved and later restored, or to be sent from one machine to another. IMP uses the [cereal library](https://uscilab.github.io/cereal/) to implement serialization in C++. In Python, the objects can be loaded or saved using the [pickle](https://docs.python.org/3/library/pickle.html#module-pickle) module.\n",
"\n",
"To add basic serialization support to our class, we first must add the cereal headers to our\n",
"[C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), `include/MyRestraint2.h`:\n",
"\n",
"```cpp\n",
"#include <cereal/access.hpp>\n",
"```\n",
"\n",
"Then we add a default constructor and a new private method `serialize` to the same [C++ header file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/include/MyRestraint2.h), which the cereal library will use to both read and write the class state:\n",
"\n",
"```cpp\n",
" MyRestraint2() {}\n",
"private:\n",
" friend class cereal::access;\n",
" template<class Archive> void serialize(Archive &ar) {\n",
" ar(cereal::base_class<Restraint>(this), p_, k_);\n",
" }\n",
" IMP_OBJECT_SERIALIZE_DECL(MyRestraint2);\n",
"```\n",
"\n",
"The default constructor (constructor which takes no arguments) is used to create an empty `MyRestraint2` object when deserializing - first the empty object is constructed, and then the class state is filled in from the serialization data.\n",
"\n",
"The class state comprises the state of the base `Restraint` class (which is handled by `cereal::base_class`), plus the particle our restraint acts on (`p_`) and the force constant (`k_`). (The `friend` declaration used here allows the cereal library to call our `serialize` method, which normally it would not be able to do since the method is marked `private`.)\n",
"\n",
"The `IMP_OBJECT_SERIALIZE_DECL` macro is used to handle polymorphic classes, which includes most IMP restraints. It needs to be paired with a similar macro in the [cpp file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/src/MyRestraint2.cpp), `src/MyRestraint2.cpp`, which uses the fully qualified name of the class:\n",
"```cpp\n",
"IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2);\n",
"```\n",
"\n",
"To add support for Python [pickle](https://docs.python.org/3/library/pickle.html#module-pickle), we replace the `IMP_SWIG_OBJECT` macro in the [SWIG interface file](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/pyext/swig.i-in), `pyext/swig.i-in`, with `IMP_SWIG_OBJECT_SERIALIZE`:\n",
"\n",
"```cpp\n",
"IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s);\n",
"```\n",
"\n",
"Serialization support should also be tested - see the `test_serialize` and `test_serialize_polymorphic` methods in the [test file at GitHub](https://github.com/salilab/imp_coding_tutorial/tree/main/modules/foo/test/test_restraint2.py)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -506,7 +592,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
"version": "3.11.4"
}
},
"nbformat": 4,
Expand Down
36 changes: 36 additions & 0 deletions modules/foo/include/MyRestraint2.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#ifndef IMPFOO_MY_RESTRAINT2_H
#define IMPFOO_MY_RESTRAINT2_H

#include <IMP/foo/foo_config.h>
#include <IMP/Restraint.h>
#include <cereal/access.hpp>

IMPFOO_BEGIN_NAMESPACE

class IMPFOOEXPORT MyRestraint2 : public Restraint {
ParticleIndex p_;
double k_;

public:
MyRestraint2(Model *m, ParticleIndex p, double k);
void do_add_score_and_derivatives(ScoreAccumulator sa) const override;
ModelObjectsTemp do_get_inputs() const override;
IMP_OBJECT_METHODS(MyRestraint2);

// RMF output support
RestraintInfo *get_static_info() const override;

// Serialization support
MyRestraint2() {}
private:
friend class cereal::access;
template<class Archive> void serialize(Archive &ar) {
ar(cereal::base_class<Restraint>(this), p_, k_);
}
IMP_OBJECT_SERIALIZE_DECL(MyRestraint2);

};

IMPFOO_END_NAMESPACE

#endif /* IMPFOO_MY_RESTRAINT2_H */
3 changes: 3 additions & 0 deletions modules/foo/pyext/swig.i-in
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

IMP_SWIG_OBJECT(IMP::foo, MyRestraint, MyRestraints);

IMP_SWIG_OBJECT_SERIALIZE(IMP::foo, MyRestraint2, MyRestraint2s);

//%include "IMP/example/ExampleRestraint.h"

%include "IMP/foo/MyRestraint.h"
%include "IMP/foo/MyRestraint2.h"
34 changes: 34 additions & 0 deletions modules/foo/src/MyRestraint2.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#include <IMP/foo/MyRestraint2.h>
#include <IMP/core/XYZ.h>

IMPFOO_BEGIN_NAMESPACE

MyRestraint2::MyRestraint2(Model *m, ParticleIndex p, double k)
: Restraint(m, "MyRestraint%1%"), p_(p), k_(k) {}

void MyRestraint2::do_add_score_and_derivatives(ScoreAccumulator sa) const {
core::XYZ d(get_model(), p_);
double score = .5 * k_ * square(d.get_z());
if (sa.get_derivative_accumulator()) {
double deriv = k_ * d.get_z();
d.add_to_derivative(2, deriv, *sa.get_derivative_accumulator());
}
sa.add_score(score);
}

ModelObjectsTemp MyRestraint2::do_get_inputs() const {
return ModelObjectsTemp(1, get_model()->get_particle(p_));
}

// RMF output support
RestraintInfo *MyRestraint2::get_static_info() const {
IMP_NEW(RestraintInfo, ri, ());
ri->add_string("type", "IMP.foo.MyRestraint2");
ri->add_float("force constant", k_);
return ri.release();
}

// Serialization support
IMP_OBJECT_SERIALIZE_IMPL(IMP::foo::MyRestraint2);

IMPFOO_END_NAMESPACE
64 changes: 64 additions & 0 deletions modules/foo/test/test_restraint2.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
from __future__ import print_function, division
import IMP
import IMP.test
import IMP.algebra
import IMP.core
import IMP.foo
import pickle


class Tests(IMP.test.TestCase):

def test_my_restraint(self):
"""Test scoring of MyRestraint2"""
m = IMP.Model()
p = m.add_particle("p")
d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3))
r = IMP.foo.MyRestraint2(m, p, 10.)
self.assertAlmostEqual(r.evaluate(True), 45.0, delta=1e-4)
self.assertLess(IMP.algebra.get_distance(d.get_derivatives(),
IMP.algebra.Vector3D(0,0,30)),
1e-4)
self.assertEqual(len(r.get_inputs()), 1)

def test_static_info(self):
"""Test static info of MyRestraint2"""
m = IMP.Model()
p = m.add_particle("p")
d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3))
r = IMP.foo.MyRestraint2(m, p, 10.)
info = r.get_static_info()
self.assertEqual(info.get_number_of_string(), 1)
self.assertEqual(info.get_string_key(0), "type")
self.assertEqual(info.get_string_value(0), "IMP.foo.MyRestraint2")

self.assertEqual(info.get_number_of_float(), 1)
self.assertEqual(info.get_float_key(0), "force constant")
self.assertAlmostEqual(info.get_float_value(0), 10.0, delta=0.001)

def test_serialize(self):
"""Test (un-)serialize of MyRestraint2"""
m = IMP.Model()
p = m.add_particle("p")
d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3))
r = IMP.foo.MyRestraint2(m, p, 10.)
self.assertAlmostEqual(r.evaluate(False), 45.0, delta=1e-3)
dump = pickle.dumps(r)
newr = pickle.loads(dump)
self.assertAlmostEqual(newr.evaluate(False), 45.0, delta=1e-3)

def test_serialize_polymorphic(self):
"""Test (un-)serialize of MyRestraint2 via polymorphic pointer"""
m = IMP.Model()
p = m.add_particle("p")
d = IMP.core.XYZ.setup_particle(m, p, IMP.algebra.Vector3D(1,2,3))
r = IMP.foo.MyRestraint2(m, p, 10.)
sf = IMP.core.RestraintsScoringFunction([r])
self.assertAlmostEqual(sf.evaluate(False), 45.0, delta=1e-3)
dump = pickle.dumps(sf)
newsf = pickle.loads(dump)
self.assertAlmostEqual(newsf.evaluate(False), 45.0, delta=1e-3)


if __name__ == '__main__':
IMP.test.main()

0 comments on commit e3322bf

Please sign in to comment.