From 5cbbea11abc542c509389737d7bd194166ad1896 Mon Sep 17 00:00:00 2001 From: Kalle Westerling Date: Fri, 3 Nov 2023 16:04:29 +0000 Subject: [PATCH] Markdown --- docs/research_ideas.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/research_ideas.md b/docs/research_ideas.md index d462d3cd..456f7690 100644 --- a/docs/research_ideas.md +++ b/docs/research_ideas.md @@ -10,7 +10,8 @@ resources in the [](resources.md) page before starting. Why not [join our Slack channel](https://docs.google.com/forms/d/e/1FAIpQLScsI8EiXDdSfn1huMp1vj5JAxi9NIeYLljbEUlMceZvwVpugw/viewform) and start a conversation around these ideas or your own? :-) -### Transfer learning from regions of dense observations to regions of sparse observations +## Transfer learning from regions of dense observations to regions of sparse observations + Since the `ConvNP` is a data-hungry model, it does not perform well if only trained on a small number of observations, which presents a challenge for modelling variables that are poorly observed. @@ -19,20 +20,23 @@ Can we train a model on a region of dense observations and then transfer the mod of sparse observations? Does the performance improve? -### Sensor placement for forecasting +## Sensor placement for forecasting + Previous active learning research with ConvNPs has only considered sensor placement for interpolation. Do the sensor placements change when the model is trained for forecasting? See, e.g., Section 4.2.1 of [Environmental sensor placement with convolutional Gaussian neural processes](https://doi.org/10.1017/eds.2023.22). -### U-Net architectural changes +## U-Net architectural changes + The `ConvNP` currently uses a vanilla U-Net architecture. Do any architectural changes improve performance, such as batch normalisation or dropout? This would require digging into the [`neuralprocesses.construct_convgnp` method](https://github.com/wesselb/neuralprocesses/blob/f20572ba480c1279ad5fb66dbb89cbc73a0171c7/neuralprocesses/architectures/convgnp.py#L97) and replacing the U-Net module with a custom one. -### Extension to continuous time observations +## Extension to continuous time observations + The `ConvNP` currently assumes that the observations are on a regular time grid. How can we extend this to continuous time observations, where the observations are not necessarily on a regular time grid? @@ -40,7 +44,8 @@ Can we do this without a major rework of the code and model? For example, can we pass a 'time of observation' auxiliary input to the model? What are the limitations of this approach? -### Training with ablations for interpretability +## Training with ablations for interpretability + Since the `ConvNP` operates on sets of observations, it is possible to ablate observations and see how the model's predictions change. Thus, the `ConvNP` admits unique interpretability opportunities. @@ -52,7 +57,8 @@ For example, when generating `Task`s with a `TaskLoader`, randomly set some of t Then, at test time, ablate context sets and measure the change in the model's predictions or performance. -### Monte Carlo sensor placement using AR sampling +## Monte Carlo sensor placement using AR sampling + The `GreedyAlgorithm` for sensor placement currently uses the model's mean prediction to infill missing observations at query sites. However, one could also draw multiple [AR samples](user-guide/prediction.ipynb)