Add a Jupyter esmvalcore API tutorial #339

flicj191 · 2024-09-06T02:12:31Z

Pull Request checklist

We appreciate your time and effort to improve the tutorial. Please keep in mind that lesson maintainers are volunteers and it may be some time before they can respond to your contribution.

Before you start

Read CONTRIBUTING.md.
Create an issue to discuss your idea. This allows your contributions to be incorporated into the tutorial.
Mention posibility of using Jupyter notebooks in the tutorial #283 and Potential lesson additions from the May 2024 Workshop discussions #324

Tasks

Give this pull request a descriptive title.
If you are contributing to existing lesson materials, please make sure the content conforms to the Lesson development section in CONTRIBUTING.md and does not contain any spelling or grammatical errors.
If you are making a new episode, please make sure the content conforms to the Lesson organization and Lesson formatting sections in CONTRIBUTING.md and does not contain any spelling or grammatical errors.
Preferably Codacy checks pass. Status can be seen below your pull request. If there is an error, click the link to find out why.
Preview changes on your machine before pushing them to GitHub by running make serve, alternatively make docker-serve. Please see the Previewing your changes locally section in CONTRIBUTING.md for installation instructions.
All code instructions have been tested.

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.

Closes {Link to the corresponding issue} #283 and part of #324

flicj191 · 2024-09-06T02:55:49Z

Adding lesson made for our ACCESS 2024 workshop Training day Hackathon with @rbeucher. Have edited to remove references to Gadi (Australia's NCI supercomputer) workflow

rbeucher · 2024-10-17T06:26:16Z

@LisaBock , @rswamina
Would any of you have time to look at this? Thank you!

LisaBock · 2024-10-17T15:44:55Z

@LisaBock , @rswamina Would any of you have time to look at this? Thank you!

Sorry! I will do next week...

rswamina · 2024-10-21T11:01:44Z

@rbeucher - I will also try it this week. Thanks!

LisaBock

Thanks a lot, @flicj191 ! It is great that we have finally a episode how to use the Jupyter esmvalcore API. And since I've never used it before, I can say that you learn it well with this episode. The examples and exercises are also very useful.

Some comments:

I am missing in the beginning a small part about the configuration. How to check the CFG keys and how to adapt them. (I think Peter has it in his blog.)
When using the ESMValCore preprocessor function I got many warnings with the output. Is it the same for you? I am not sure if we should mention that somehow. Otherwise it could be irritating.
I could not find the 'ACCESS-OM2, omip2' data on ESGF. Did I something wrong or could you replace it with a dataset which is available on ESGF?

flicj191 · 2024-10-24T00:01:45Z

Thanks @LisaBock for your comments! Good points, I will address them though I'm at a conference next week so probably the week after. Cheers

rswamina · 2024-11-05T21:45:45Z

@flicj191 - I have started this review..will try and see if the instructions work simultaneously.

rswamina · 2024-11-19T15:38:49Z

_episodes/11-esmvalcoreapi.md

@@ -0,0 +1,511 @@
+---
+title: "ESMValCore API in a Jupyter notebook"


In the tutorial so far, we have not distinguished between ESMvalCore and ESMValTool. Also we have not referred to the ESMValCore API as such, so perhaps we should write a line or two to orient the user.

I have extended the title and added more to the start of the episode

rswamina · 2024-11-19T15:44:04Z

_episodes/11-esmvalcoreapi.md

+you can run code.
+You will need to use a python environment with ESMValTool and ESMValCore installed.
+
+## Find Datasets with facets


Could we define facets before this?

I have added some more words to define and referred to another section in a previous episode

rswamina · 2024-11-19T15:46:01Z

_episodes/11-esmvalcoreapi.md

+You will need to use a python environment with ESMValTool and ESMValCore installed.
+
+## Find Datasets with facets
+We have seen from running available recipes that ESMValTool is able to find data from facets that


Can we provide an example for this? I think we either need to have a lesson referencing facets before this or we should adda paragraph with examples here.

edited as above comment, maybe we could add for information to a previous lesson?

rswamina · 2024-11-19T15:51:07Z

_episodes/11-esmvalcoreapi.md

+were given in the recipe. We can use this in a Notebook, including filling out the facets for 
+data definition. 
+To do this we will use the `Dataset` object from the API. Let's look at this example. 
+


We usually provide guidance on where this code should go. For instance, is this in a Notebook or a Python file or a YAML file? As it stands, it is not clear where this code will go and what it will be used for. Partly because we haven't discussed ESMValCore APIs in the tutorial.

I have mentioned adding to a jupyter notebook

rswamina · 2024-11-19T16:19:03Z

_episodes/11-esmvalcoreapi.md

+## Start JupyterLab
+A [jupyter notebook](https://jupyter.org/){:target="_blank"} is an interactive document where 
+you can run code.
+You will need to use a python environment with ESMValTool and ESMValCore installed.


We will need to tell them how to do this on one server (minimally) and ideally provide steps for their own machine and one or more HPC servers such as JASMIN.

I'm not sure we can go to too much detail as support would be at the HPC centers and I wouldn't know the others that well. I have added in links to them. On their own machine we could also link to the jupyter intall page

rswamina · 2024-11-19T16:19:56Z

_episodes/11-esmvalcoreapi.md

+print(dataset)
+```
+> ## Pro tip: Augmented facets in the output
+> When running a recipe there is a `_filled` recipe `yml` file in the output `/run` folder which


We would have to provide the command for running this recipe.

added and changed example recipe

rswamina · 2024-11-19T16:31:56Z

@flicj191 - This is really nice! I have added some comments at the start of the lesson essentially to provide more detailed instructions in keeping with other lessons and for easy assimilation by the user. We haven't mentioned anything about the ESMValCore API in the tutorial, so either we should provide some background here and/or link to documentation in a way that explains this.
Also, we have to have some way of explaining how to use the appropriate environment.

I'd be interested to know what you and @LisaBock think about this.

rswamina

@flicj191 - Leaving this request for changes based on overall comments. Happy to discuss how best to make these changes.

flicj191 · 2024-11-26T05:27:22Z

Hi @LisaBock and @rswamina I have edited this to address all your comments in this episode. Let me know what you think, thanks

flicj191 added 5 commits September 6, 2024 11:58

add core api tutorial

ade4ad2

edit sea-ice area exercise

e002f43

clean up

aaade54

shorten lines

fddd484

clean up checks

6f4e6c7

flicj191 requested review from LisaBock and rswamina September 6, 2024 02:57

LisaBock requested changes Oct 22, 2024

View reviewed changes

rswamina reviewed Nov 19, 2024

View reviewed changes

rswamina requested changes Nov 19, 2024

View reviewed changes

flicj191 added 4 commits November 21, 2024 22:07

episode update

875e3a1

Add to episode addressing cooments

4078a54

add link to JASMIN

60f2a15

reduce line lengths

97f0a8d

flicj191 requested review from rswamina and LisaBock November 26, 2024 05:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a Jupyter esmvalcore API tutorial #339

Add a Jupyter esmvalcore API tutorial #339

flicj191 commented Sep 6, 2024 •

edited

Loading

flicj191 commented Sep 6, 2024

rbeucher commented Oct 17, 2024

LisaBock commented Oct 17, 2024

rswamina commented Oct 21, 2024

LisaBock left a comment

flicj191 commented Oct 24, 2024

rswamina commented Nov 5, 2024

rswamina Nov 19, 2024

flicj191 Nov 26, 2024

rswamina Nov 19, 2024

flicj191 Nov 26, 2024

rswamina Nov 19, 2024 •

edited

Loading

flicj191 Nov 26, 2024

rswamina Nov 19, 2024

flicj191 Nov 26, 2024

rswamina Nov 19, 2024

flicj191 Nov 26, 2024

rswamina Nov 19, 2024

flicj191 Nov 26, 2024

rswamina commented Nov 19, 2024

rswamina left a comment

flicj191 commented Nov 26, 2024

		@@ -0,0 +1,511 @@
		---
		title: "ESMValCore API in a Jupyter notebook"

Add a Jupyter esmvalcore API tutorial #339

Are you sure you want to change the base?

Add a Jupyter esmvalcore API tutorial #339

Conversation

flicj191 commented Sep 6, 2024 • edited Loading

Pull Request checklist

Before you start

Tasks

flicj191 commented Sep 6, 2024

rbeucher commented Oct 17, 2024

LisaBock commented Oct 17, 2024

rswamina commented Oct 21, 2024

LisaBock left a comment

Choose a reason for hiding this comment

flicj191 commented Oct 24, 2024

rswamina commented Nov 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rswamina Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rswamina commented Nov 19, 2024

rswamina left a comment

Choose a reason for hiding this comment

flicj191 commented Nov 26, 2024

flicj191 commented Sep 6, 2024 •

edited

Loading

rswamina Nov 19, 2024 •

edited

Loading