Markdown

docs/research_ideas.md

@@ -10,7 +10,8 @@ resources in the [](resources.md) page before starting.
 Why not [join our Slack channel](https://docs.google.com/forms/d/e/1FAIpQLScsI8EiXDdSfn1huMp1vj5JAxi9NIeYLljbEUlMceZvwVpugw/viewform)
 and start a conversation around these ideas or your own? :-)
 
-### Transfer learning from regions of dense observations to regions of sparse observations
+## Transfer learning from regions of dense observations to regions of sparse observations
+
 Since the `ConvNP` is a data-hungry model, it does not perform well if only trained on a
 small number of observations, which presents a challenge for modelling variables that
 are poorly observed.
@@ -19,28 +20,32 @@ Can we train a model on a region of dense observations and then transfer the mod
 of sparse observations?
 Does the performance improve?
 
-### Sensor placement for forecasting
+## Sensor placement for forecasting
+
 Previous active learning research with ConvNPs has only considered sensor placement for interpolation.
 Do the sensor placements change when the model is trained for forecasting?
 
 See, e.g., Section 4.2.1 of [Environmental sensor placement with convolutional Gaussian neural processes](https://doi.org/10.1017/eds.2023.22).
 
-### U-Net architectural changes
+## U-Net architectural changes
+
 The `ConvNP` currently uses a vanilla U-Net architecture.
 Do any architectural changes improve performance, such as batch normalisation or dropout?
 
 This would require digging into the [`neuralprocesses.construct_convgnp` method](https://github.com/wesselb/neuralprocesses/blob/f20572ba480c1279ad5fb66dbb89cbc73a0171c7/neuralprocesses/architectures/convgnp.py#L97)
 and replacing the U-Net module with a custom one.
 
-### Extension to continuous time observations
+## Extension to continuous time observations
+
 The `ConvNP` currently assumes that the observations are on a regular time grid.
 How can we extend this to continuous time observations, where the observations are not necessarily
 on a regular time grid?
 Can we do this without a major rework of the code and model?
 For example, can we pass a 'time of observation' auxiliary input to the model?
 What are the limitations of this approach?
 
-### Training with ablations for interpretability
+## Training with ablations for interpretability
+
 Since the `ConvNP` operates on sets of observations, it is possible to ablate observations
 and see how the model's predictions change.
 Thus, the `ConvNP` admits unique interpretability opportunities.
@@ -52,7 +57,8 @@ For example, when generating `Task`s with a `TaskLoader`, randomly set some of t
 Then, at test time, ablate context sets and measure the change in the model's predictions
 or performance.
 
-### Monte Carlo sensor placement using AR sampling
+## Monte Carlo sensor placement using AR sampling
+
 The `GreedyAlgorithm` for sensor placement currently uses the model's mean prediction
 to infill missing observations at query sites.
 However, one could also draw multiple [AR samples](user-guide/prediction.ipynb)