Add future work documentation

docs/future-work/cmp-hierarchy-improvements.md

@@ -4,9 +4,8 @@ description: Improvements we would like to add to the CMP or hierarchical inform
 ---
 These are the things we would like to implement:
 
-- [Figure out performance measure and supervision in heterarchy](cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md) #infrastructure
-- [Add top-down connections](cmp-hierarchy-improvements/add-top-down-connections.md) #numsteps
-- [Add associative connections](cmp-hierarchy-improvements/add-associative-connections.md) #abstract #numsteps
+- [Figure out performance measures and supervision in heterarchy](cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md) #infrastructure #compositional
+- [Add top-down connections](cmp-hierarchy-improvements/add-top-down-connections.md) #numsteps #multiobj #compositional
 - [Run & Analyze experiments with >2LMs in heterarchy testbed](cmp-hierarchy-improvements/run-analyze-experiments-with-2lms-in-heterarchy-testbed.md) #compositional
 - [Run & Analyze experiments in multiobject environment looking at scene graphs](cmp-hierarchy-improvements/run-analyze-experiments-in-multiobject-environment-looking-at-scene-graphs.md) #multiobj
 - [Test learning at different speeds depending on level in hierarchy](cmp-hierarchy-improvements/test-learning-at-different-speeds-depending-on-level-in-hierarchy.md) #learning #generalization

docs/future-work/cmp-hierarchy-improvements/add-top-down-connections.md

@@ -1,3 +1,9 @@
 ---
 title: Add Top-Down Connections
 ---
+
+In Monty systems, low-level LMs project to high-level LMs, where this projection occurs if their sensory receptive fields are co-aligned. Hierarchical connections should be able to learn a mapping between objects represented at these low-level LMs, and objects represented in the high-level LMs that frequently co-occur. Such learning would be similar to that required for [Generalizing Voting To Associative Connections](../voting-improvements/generalize-voting-to-associative-connections.md).
+
+For example, a high-level LM of a dinner-set might have learned that the fork is present at a particular location in its internal reference frame. When at that location, it would therefore predict that the low-level LM should be sensing a fork, enabling the perception of a fork in the low-level LM even when there is a degree of noise or other source of uncertainty in the low-level LM's representation.
+
+In the brain, these top-down projections correspond to L6 to L1 connections, where the synapses at L1 would support predictions about object ID. However, these projections also form local synapses en-route through the L6 layer of the lower-level cortical column. In a Monty LM, this would correspond to the top-down connection predicting not just the object that the low-level LM should be sensing, but also the specific location that it should be sensing it at. This could be complemented with predicting a particular pose of the low-level object (see [Use Better Priors for Hypothesis Initialization](../learning-module-improvements/use-better-hypothesis-priors.md)).

docs/future-work/cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md

@@ -1,3 +1,10 @@
 ---
-title: Figure out Performance Measure and Supervision in Heterarchy
+title: Figure out Performance Measures and Supervision in Heterarchy
 ---
+As we introduce hierarchy and compositional objects, such as a dinner-table setting, we need to figure out both how to measure the performance of the system, and how to supervise the learning. For the latter, we might choose to train the system on component objects in isolation (a fork, a knife, etc.) before then showing Monty the full compositional object (the dinner-table setting). When evaluating performance, we might then see how well the system retrieves representations at different levels of the hierarchy. However, in the more core setting of unsupervised learning, representations of the sub-objects would likely also emerge at the high level (a coarse knife representation, etc.), while we may also find some representations of the dinner scene in low-level LMs. Deciding then how we measure performance will be more difficult.
+
+When we move to objects with less obvious composition (i.e. where the sub-objects must be disentangled in a fully unsupervised manner), representations will emerge at different levels of the system that may not correspond to any labels present in our datasets. For example, handles, or the head of a spoon, may emerge as object-representations in low-level LMs, even though the dataset only recognizes labels like "mug" and "spoon".
+
+This is less clear, but one approach to measure the "correctness" of representations in this setting might be how well a predicted representation aligns with the outside world. For example, while LMs are not designed to be used as generative models, we could visualize how well an inferred object graph maps onto the object actually present in the world. Quantifying such alignment might leverage measures such as differences in point-clouds. This would provide some evidence of how well the learned decomposition of objects corresponds to the actual objects present in the world.
+
+See also [Make Dataset to Test Compositional Objects](../environment-improvements/make-dataset-to-test-compositional-objects.md) and [Metrics to Evaluate Categories and Generalization](../environment-improvements/create-dataset-and-metrics-to-evaluate-categories-and-generalization.md).

docs/future-work/cmp-hierarchy-improvements/send-similarity-encoding-object-id-to-next-level-test.md

@@ -1,3 +1,17 @@
 ---
 title: Send Similarity Encoding Object ID to Next Level & Test
 ---
+
+We have implemented the ability to encode object IDs using sparse-distributed representations (SDRs), and in particular can use this as a way of capturing similarity and disimlarity between objects. Using such encodings in learned [Hierarchical Connections](add-top-down-connections.md), we should observe a degree of natural generalization when recognizing compositional objects.
+
+For example, assume a Monty system learns a dinner table setting with normal cuttlery and plates (see examples below). Separately, the system learns about medieval instances of cuttlery and plates, but never sees them arranged in a dinner table setting. Based on the similarity of the medieval cutterly objects to their modern counterparts, the objects should have considerable overlap in their SDR encodings.
+
+If the system was to then see a medieval dinner table setting for the first time, it should be able to recognize the arrangement as a dinner-table setting with reasonable confidence, even if the constituent objects are somewhat different from those present when the compositional object was first learned.
+
+We should note that we are still determining whether overlapping bits between SDRs is the best way to encode object similarity. As such, we are also open to exploring this task with alternative approaches, such as directly making use of values in the evidence-similarity matrix (from which SDRs are currently derived).
+
+![Standard dinner table setting](../../figures/future-work/dinner_standard.png)
+*Example of a standard dinner table setting with modern cutlery and plates that the system could learn from.*
+
+![Medieval dinner table setting](../../figures/future-work/dinner_medieval.png)
+*Example of a medieval dinner table setting with medieval cutlery and plates that the system could be evaluated on, after having observed the individual objects in isolation.*

docs/future-work/cmp-hierarchy-improvements/test-learning-at-different-speeds-depending-on-level-in-hierarchy.md

@@ -1,3 +1,13 @@
 ---
 title: Test Learning at Different Speeds Depending on Level in Hierarchy
 ---
+
+Our general view is that episodic memory and working memory in the brain leverage similar representations to those in learning modules, i.e. structured reference frames of discrete objects.
+
+For example, the brain has a specialized region for episodic memory (the hippocampal complex), due to the large number of synapses required to rapidly form novel binding associations. However, we believe the core algorithms of the hippocampal complex follow the same principles of a cortical column (and therefore a learning module), with learning simply occurring on a faster time scale.
+
+As such, we would like to explore adding forms of episodic and working memory by introducing high-level learning modules that learn information on extremely fast time scales relative to lower-level LMs. These should be particularly valuable in settings such as recognizing multi-object arrangements in a scene, and providing memory when a Monty system is performing a multi-step task. Note that because of the overlap in the core algorithms, LMs can be used largely as-is for these memory systems, with the only change being the learning rate.
+
+It is worth noting that the `GridObjectModel` would be particularly well suited for introducing a learning-rate parameter, due to its constraints on the amount of information that can be stored.
+
+As a final note, varying the learning rate across learning modules will likely play an important role in dealing with representational drift, and the impact it can have on continual learning. For example, we expect that low-level LMs, which partly form the representations in higher-level LMs, will change their representations more slowly.

docs/future-work/environment-improvements.md

@@ -5,7 +5,7 @@ description: New environments and benchmark experiments we would like to add.
 
 These are the things we would like to implement:
 
-- [Make dataset to test compositional objects](environment-improvements/make-dataset-to-test-compositional-objects.md) #compositional
+- [Make dataset to test compositional objects](environment-improvements/make-dataset-to-test-compositional-objects.md) #compositional #multiobj
 - [Set up Environment that allows for object manipulation](environment-improvements/set-up-environment-that-allows-for-object-manipulation.md) #goalpolicy
 - [Set up object manipulation benchmark tasks and evaluation measures](environment-improvements/set-up-object-manipulation-benchmark-tasks-and-evaluation-measures.md) #goalpolicy
 - [Create dataset and metrics to evaluate categories and generalization](environment-improvements/create-dataset-and-metrics-to-evaluate-categories-and-generalization.md) #generalization

docs/future-work/environment-improvements/create-dataset-and-metrics-to-evaluate-categories-and-generalization.md

@@ -1,3 +1,11 @@
 ---
 title: Create Dataset and Metrics to Evaluate Categories and Generalization
 ---
+
+Datasets do not typically capture the flexibility of object labels based on whether an object belongs to a broad class (e.g. cans), vs. a specific instance of a class (e.g. a can of tomato soup).
+
+Labeling a dataset with "hierarchical" labels, such that an object might be both a "can", as well as a "can of tomato soup" would be one approach to capturing this flexibility. Once available, classification accuracy could be assessed both at the level of individual object instances, as well as at the level of categories.
+
+We might leverage crowd-sourced labels to ensure that this labeling is reflective of human perception, and not biased by our beliefs as designers of Monty. This also relates to the general problem fo [Multi-Label Classification](https://paperswithcode.com/task/multi-label-classification), and so there may be off-the-shelf solutions that we can explore.
+
+Initially such labels should focus on morphology, as this is the current focus of Monty's recognition system. However, we would eventually want to also account for affordances, such as an object that is a chair, a vessel, etc. Being able to classify objects based on their affordances would be an experimental stepping stone to the true measure of the systems representations, which would be how well affordances are used to manipulate the world.

docs/future-work/environment-improvements/make-dataset-to-test-compositional-objects.md

@@ -1,3 +1,15 @@
 ---
 title: Make Dataset to Test Compositional Objects
 ---
+
+We have developed an initial dataset based on recognizing a variety of dinner table sets with different arrangements of plates and cutlery. For example, the objects can be arranged in a normal setting, or aligned in a row (i.e. not a typical dinner-table setting). Similarly, the component objects can be those of a modern dining table, or those from a "medieval" time-period. As such, this dataset can be used to test the ability of Monty systems to recognize compositional objects based on the specific arrangement of objects, and to test generalization to novel compositions.
+
+By using explicit objects to compose multi-part objects, this dataset has the advantage that we can learn on the component objects in isolation, using supervised learning signals if necessary. It's worth noting that this is often how learning of complex compositional objects takes place in humans. For example, when learning to read, children begin by learning individual letters, which are themselves composed of a variety of strokes. Only when letters are learned can they learn to combine them into words. More generally, disentangling an object from other objects is difficult without the ability to interact with it, or see it in a sufficient range of contexts that it's separation from other objects becomes clear.
+
+However, we would eventually expect compositional objects to be learned in an unsupervised manner. When this is consistently possible, we can consider more diverse datasets where the component objects may not be as explicit. At that time, the challenges described in [Figure out Performance Measure and Supervision in Heterarchy](../cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md) will become more relevant.
+
+![Dinner table set](../../figures/future-work/dinner_variations_standard.png)
+*Example of compositional objects made up of modern cutlery and plates.*
+
+![Dinner table set](../../figures/future-work/dinner_variations_medieval.png)
+*Example of compositional objects made up of medieval cutlery and plates.*

docs/future-work/environment-improvements/set-up-environment-that-allows-for-object-manipulation.md

@@ -1,3 +1,7 @@
 ---
 title: Set up Environment that Allows for Object Manipulation
 ---
+
+See [Decompose Goals Into Subgoals & Communicate](../motor-system-improvements/decompose-goals-into-subgoals-communicate.md) for a discussion of the kind of tasks we are considering for early object-manipulation experiments. An even simpler task that we have recently considered is pressing a switch to turn a lamp on or off. We will provide further details on what these tasks might look like soon.
+
+Beyond the specifics of any given task, an important part of this future-work component is to identify a good simulator for such settings. For example, we would like to have a setup where objects are subject to gravity, but are prevented from falling into a void by a table or floor. Other elements of physics such as friction should also be simulated, while it should be straightforward to reset an environment, and specify the arrangement of objects (for example using 3D modelling software).

docs/future-work/framework-improvements.md

@@ -4,7 +4,7 @@ description: Improvements we would like to make on the general code framework.
 ---
 These are the things we would like to implement:
 
-- [Add infrastructure for multiple agents that move independently](framework-improvements/add-infrastructure-for-multiple-agents-that-move-independently.md) #numsteps #infrastructure
+- [Add infrastructure for multiple agents that move independently](framework-improvements/add-infrastructure-for-multiple-agents-that-move-independently.md) #numsteps #infrastructure #goalpolicy
 - [Automate benchmark experiments & analysis](framework-improvements/automate-benchmark-experiments-analysis.md) #infrastructure
 - [Add more wandb logging for learning from unsupervised](framework-improvements/add-more-wandb-logging-for-learning-unsupervised.md) #learning
 - [Add GPU support for Monty](framework-improvements/add-gpu-support-for-monty.md) #speed

docs/future-work/framework-improvements/add-infrastructure-for-multiple-agents-that-move-independently.md

@@ -1,3 +1,11 @@
 ---
 title: Add Infrastructure for Multiple Agents that Move Independently
 ---
+
+Currently, Monty's infrastructure only supports a single agent that moves around the scene, where that agent can be associated with a plurality of sensors and LMs. We would like to add support for multiple agents that move independently.
+
+For example, a hand-like surface-agent might explore the surface of an object, where each one of its "fingers" can move in a semi-independent manner. At the same time, a distant-agent might observe the object, saccading across its surface independent of the surface agent. At other times they might coordinate, such that they perceive the same location on an object at the same time, which would be useful while voting connections are still being learned (see [Generalize Voting to Associative Connections](../voting-improvements/generalize-voting-to-associative-connections.md)).
+
+An example of a first task that could make use of this infrastructure is [Implement a Simple Cross-Modal Policy for Sensory Guidance](../motor-system-improvements/simple-cross-modal-policy.md).
+
+It's also worth noting that we would like to move towards the concept of "motor modules" in the code-base, i.e. a plurarity of motor modules that convert from CMP-compliant goal states to non-CMP actuator changes. This would be a shift from the singular "motor system" that we currently have.

docs/future-work/learning-module-improvements.md

@@ -9,13 +9,16 @@ These are the things we would like to implement:
 - [Use off-object observations](learning-module-improvements/use-off-object-observations.md) #numsteps #multiobj
 - [Reinitialize hypotheses when starting to recognize a new object](learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md) #multiobj
 - [Improve bounded evidence performance](learning-module-improvements/improve-bounded-evidence-performance.md) #multiobj
-- [Use models with less points](learning-module-improvements/use-models-with-less-points.md) #speed #generalization
+- [Use models with fewer points](learning-module-improvements/use-models-with-fewer-points.md) #speed #generalization
 - [Make it possible to store multiple feature maps on one graph](learning-module-improvements/make-it-possible-to-store-multiple-feature-maps-on-one-graph.md) #featsandmorph
 - [Test particle-filter-like resampling of hypothesis space](learning-module-improvements/test-particle-filter-like-resampling-of-hypothesis-space.md) #accuracy #speed
-- [Re-anchor hypotheses](learning-module-improvements/re-anchor-hypotheses.md) #deformations #noise #generalization
+- [Re-anchor hypotheses for robustness to noise and distortions](learning-module-improvements/re-anchor-hypotheses.md) #deformations #noise #generalization
 - [Less dependency on first observation](learning-module-improvements/less-dependency-on-first-observation.md) #noise #multiobj
 - [Deal with incomplete models](learning-module-improvements/deal-with-incomplete-models.md) #learning
 - [Implement & test GNNs to model object behaviors & states](learning-module-improvements/implement-test-gnns-to-model-object-behaviors-states.md) #dynamic
 - [Deal with moving objects](learning-module-improvements/deal-with-moving-objects.md) #dynamic #realworld
+- [Support scale invariance](learning-module-improvements/support-scale-invariance.md) #scale
+- [Improve handling of symmetry](learning-module-improvements/improve-handling-of-symmetry.md) #pose
+- [Use Better Priors for Hypothesis Initialization](learning-module-improvements/use-better-hypothesis-priors.md) #numsteps #pose #scale
 
 Please see the [instructions here](project-roadmap.md#how-you-can-contribute) if you would like to tackle one of these tasks.