-
Notifications
You must be signed in to change notification settings - Fork 18
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 97b9b22
Showing
269 changed files
with
27,962 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 35dbdb2b27cad3442b8a3d7e176c0c97 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
========================= | ||
Developer Code of Conduct | ||
========================= | ||
|
||
.. | ||
TODO: write a code of conduct for DeepSensor/mirror the CODE-OF-CONDUCT.md document? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
========================== | ||
Contributing to DeepSensor | ||
========================== | ||
|
||
.. | ||
TODO: write a intro to contributing to DeepSensor/mirror the CONTRIBUTE.md document? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
============= | ||
Community FAQ | ||
============= | ||
|
||
This FAQ aims to answer common questions about the DeepSensor library. It is our way to streamline the onboarding process and clarify expectations. | ||
|
||
.. note:: | ||
|
||
If you have a question that is not answered here, please open an issue or submit a pull request. | ||
|
||
Questions | ||
--------- | ||
|
||
**Q: What is the purpose of this project?** | ||
|
||
**Answer:** [Briefly describe the project, its objectives, and its intended audience.] | ||
|
||
--- | ||
|
||
**Q: How can I contribute?** | ||
|
||
**Answer:** There are many ways to contribute, from writing code and fixing bugs to improving documentation or translating content. Check our :doc:`contribution guide <contributing>` for detailed steps. | ||
|
||
--- | ||
|
||
**Q: Where can I find the contribution guidelines?** | ||
|
||
**Answer:** You can find our contribution guidelines in the CONTRIBUTING.md file in the root directory of the repository, or in the :doc:`contribution guide <contributing>`. | ||
|
||
--- | ||
|
||
**Q: Do I need to sign a Contributor License Agreement (CLA)?** | ||
|
||
**Answer:** At the current time, we do not require a CLA from our contributors. | ||
|
||
--- | ||
|
||
**Q: How do I report a bug?** | ||
|
||
**Answer:** Please submit an issue in our GitHub repository. Make sure to provide detailed information, including steps to reproduce the bug and the expected outcome. | ||
|
||
--- | ||
|
||
**Q: How do I request a new feature?** | ||
|
||
**Answer:** Open a new issue on our GitHub repository and label it as a feature request. Describe the feature in detail and its potential benefits. | ||
|
||
--- | ||
|
||
**Q: How do I set up the development environment?** | ||
|
||
**Answer:** Follow the instructions in our developer documentation. If you run into issues, ask in our community chat (on Slack) or :doc:`contact the core group of maintainers directly </contact>`. | ||
|
||
--- | ||
|
||
**Q: Do you have a code of conduct?** | ||
|
||
**Answer:** Yes, we value a respectful and inclusive community. Please read our :doc:`Code of Conduct <code-of-conduct>` before contributing. | ||
|
||
--- | ||
|
||
**Q: How can I get in touch with other contributors or maintainers?** | ||
|
||
**Answer:** Join our Slack team to stay in touch with other contributors and maintainers. We also have a standing community meeting, which is by invite, so :doc:`get in touch with the core group of maintainers </contact>` to receive an invite to them. | ||
|
||
--- | ||
|
||
**Q: Can I contribute even if I'm not a coder?** | ||
|
||
**Answer:** Absolutely! Contributions can be made in the form of documentation, design, testing, and more. Everyone's skills are valuable. Join our Slack discussion to learn more. | ||
|
||
--- | ||
|
||
**Q: How do I claim an issue to work on?** | ||
|
||
**Answer:** Comment on the issue expressing your interest to help out. If the issue is unassigned, a maintainer will likely assign it to you. | ||
|
||
--- | ||
|
||
**Q: What's the process for proposing a significant change?** | ||
|
||
**Answer:** For significant changes, it's a good practice to first open a discussion or issue to gather feedback. Once there's a consensus, you can proceed with a pull request. | ||
|
||
--- | ||
|
||
**Q: How can I get my pull request (PR) merged?** | ||
|
||
**Answer:** Ensure your PR follows the contribution guidelines, passes all tests, and has been reviewed by at least one maintainer. Address any feedback provided. | ||
|
||
--- | ||
|
||
**Q: What's the project's release cycle?** | ||
|
||
**Answer:** We release updates in a rolling manner, following our roadmap planning. Critical bug fixes might be released as needed. | ||
|
||
--- | ||
|
||
**Q: How is credit given to contributors?** | ||
|
||
**Answer:** Contributors are acknowledged in our release notes, and their contributions are forever recorded in the project's history. | ||
|
||
|
||
.. | ||
TODO: write more questions and answers here. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
=========================================== | ||
DeepSensor's user and contributor community | ||
=========================================== | ||
|
||
The DeepSensor community is a group of users and contributors who are interested in the development of DeepSensor. The community is open to anyone who is interested in DeepSensor. The community is a place to ask questions, discuss ideas, and share your work. | ||
|
||
If you are interested in joining the community, please join our Slack channel at https://deepsensor.slack.com. You can request an invitation to the Slack channel at `https://ai4environment.slack.com/signup`. | ||
|
||
We welcome contributions from the community. If you are interested in contributing to DeepSensor, please read the :doc:`contributing` guide. | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
:caption: Table of contents: | ||
|
||
faq | ||
contributing | ||
code-of-conduct | ||
roadmap |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
================== | ||
DeepSensor Roadmap | ||
================== | ||
|
||
.. | ||
TODO: write an intro to the roadmap for DeepSensor + link to the GitHub project management board |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Contact the developers | ||
====================== | ||
|
||
[email protected] | ||
|
||
.. | ||
TODO: Add contact information here... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
==================================== | ||
Extending DeepSensor with new models | ||
==================================== | ||
|
||
To extend DeepSensor with a new model, simply create a new class that inherits from ``deepsensor.model.DeepSensorModel`` and implement the low-level prediction methods defined in ``deepsensor.model.ProbabilisticModel``, such as ``.mean`` and ``.stddev``. | ||
|
||
In this example, we'll create a new model called ``ExampleModel``: | ||
|
||
.. code-block:: python | ||
class ExampleModel(DeepSensorModel): | ||
""" | ||
A very naive model that predicts the mean of the first context set | ||
with a fixed stddev. | ||
""" | ||
def __init__(self, data_processor: DataProcessor, task_loader: TaskLoader): | ||
# Initiate the parent class (DeepSensorModel) with the | ||
# provided data processor and task loader: | ||
super().__init__(data_processor, task_loader) | ||
def mean(self, task: Task): | ||
"""Compute mean at target locations""" | ||
return np.mean(task["Y_c"][0]) | ||
def stddev(self, task: Task): | ||
"""Compute stddev at target locations""" | ||
return 0.1 | ||
... | ||
After creating ``ExampleModel`` in this way, it can be used in the same way as the built-in :class:`~deepsensor.model.convnp.ConvNP` model. | ||
|
||
See `this Jupyter notebook <https://github.com/tom-andersson/deepsensor_gallery/blob/main/demonstrators/extending_models.ipynb>`_ for more details. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
=============== | ||
Getting started | ||
=============== | ||
|
||
In this part of the documentation, we will show you how to get started with ``DeepSensor``. We will show you how to install ``DeepSensor`` and how to extend it. We will also provide you with some tutorials to get you started with ``DeepSensor``. | ||
|
||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:caption: Table of contents: | ||
|
||
installation | ||
extending | ||
tutorials/index |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
Installation instructions | ||
========================= | ||
|
||
DeepSensor is a Python package that can be installed in a number of ways. In this section we will describe the two main ways to install the package. | ||
|
||
Install from `PyPI <https://pypi.org/project/deepsensor/>`_ | ||
----------------------------------------------------------- | ||
|
||
If you want to use the latest stable release of DeepSensor and do not want/need access to the worked examples or the package's source code, we recommend installing from PyPI. | ||
|
||
This is the easiest way to install DeepSensor. | ||
|
||
- Install ``deepsensor``: | ||
|
||
.. code-block:: bash | ||
pip install deepsensor | ||
- Install the backend of your choice: | ||
|
||
- Install ``tensorflow``: | ||
|
||
.. code-block:: bash | ||
pip install tensorflow | ||
- Install ``pytorch``: | ||
|
||
.. code-block:: bash | ||
pip install torch | ||
Install from `source <https://github.com/tom-andersson/deepsensor>`_ | ||
--------------------------------------------------------------------- | ||
|
||
.. note:: | ||
|
||
You will want to use this method if you intend on contributing to the source code of DeepSensor. | ||
|
||
If you want to keep up with the latest changes to DeepSensor, or want/need easy access to the worked examples or the package's source code, we recommend installing from source. | ||
|
||
This method will create a ``DeepSensor`` directory on your machine which will contain all the source code, docs and worked examples. | ||
|
||
- Clone the repository: | ||
|
||
.. code-block:: bash | ||
git clone | ||
- Install ``deepsensor``: | ||
|
||
.. code-block:: bash | ||
pip install -e -v . | ||
- Install the backend of your choice: | ||
|
||
- Install ``tensorflow``: | ||
|
||
.. code-block:: bash | ||
pip install tensorflow | ||
- Install ``pytorch``: | ||
|
||
.. code-block:: bash | ||
pip install torch |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
========= | ||
Tutorials | ||
========= | ||
|
||
.. toctree:: | ||
:maxdepth: 2 | ||
:caption: Tutorials: | ||
|
||
quickstart |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
==================== | ||
Tutorial: Quickstart | ||
==================== | ||
|
||
Here we will demonstrate a simple example of training a convolutional conditional neural process (ConvCNP) to spatially interpolate ERA5 data. | ||
|
||
We can go from imports to predictions with a trained model in less than 30 lines of code! | ||
|
||
.. code-block:: python | ||
import deepsensor.torch | ||
from deepsensor.data.loader import TaskLoader | ||
from deepsensor.data.processor import DataProcessor | ||
from deepsensor.model.convnp import ConvNP | ||
from deepsensor.train.train import train_epoch | ||
import xarray as xr | ||
import pandas as pd | ||
import numpy as np | ||
# Load raw data | ||
ds_raw = xr.tutorial.open_dataset("air_temperature") | ||
# Normalise data | ||
data_processor = DataProcessor(x1_name="lat", x1_map=(15, 75), x2_name="lon", x2_map=(200, 330)) | ||
ds = data_processor(ds_raw) | ||
# Set up task loader | ||
task_loader = TaskLoader(context=ds, target=ds) | ||
# Set up model | ||
model = ConvNP(data_processor, task_loader) | ||
# Generate training tasks with up to 10% of grid cells passed as context and all grid cells | ||
# passed as targets | ||
train_tasks = [] | ||
for date in pd.date_range("2013-01-01", "2014-11-30")[::7]: | ||
task = task_loader(date, context_sampling=np.random.uniform(0.0, 0.1), target_sampling="all") | ||
train_tasks.append(task) | ||
# Train model | ||
for epoch in range(10): | ||
train_epoch(model, train_tasks, progress_bar=True) | ||
# Predict on new task with 10% of context data and a dense grid of target points | ||
test_task = task_loader("2014-12-31", 0.1) | ||
mean_ds, std_ds = model.predict(test_task, X_t=ds_raw) | ||
After training, the model can predict directly to `xarray` in your data's original units and coordinate system: | ||
|
||
.. code-block:: python | ||
>>> mean_ds | ||
<xarray.Dataset> | ||
Dimensions: (time: 1, lat: 25, lon: 53) | ||
Coordinates: | ||
* time (time) datetime64[ns] 2014-12-31 | ||
* lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0 | ||
* lon (lon) float32 200.0 202.5 205.0 207.5 ... 322.5 325.0 327.5 330.0 | ||
Data variables: | ||
air (time, lat, lon) float32 246.7 244.4 245.5 ... 290.2 289.8 289.4 | ||
We can also predict directly to `pandas` containing a timeseries of predictions at off-grid locations | ||
by passing a `numpy` array of target locations to the `X_t` argument of `.predict`: | ||
|
||
.. code-block:: python | ||
# Predict at two off-grid locations for three days in December 2014 | ||
test_tasks = task_loader(pd.date_range("2014-12-01", "2014-12-31"), 0.1) | ||
mean_df, std_df = model.predict(test_tasks, X_t=np.array([[50, 280], [40, 250]]).T) | ||
.. code-block:: python | ||
>>> mean_df | ||
air | ||
time lat lon | ||
2014-12-01 50.0 280.0 260.183056 | ||
40.0 250.0 277.947373 | ||
2014-12-02 50.0 280.0 261.08943 | ||
40.0 250.0 278.219599 | ||
2014-12-03 50.0 280.0 257.128185 | ||
40.0 250.0 278.444229 | ||
This quickstart example is also `available as a Jupyter notebook <https://github.com/tom-andersson/deepsensor_demos/blob/main/demonstrators/quickstart.ipynb>`_ with added visualisations. |
Oops, something went wrong.