Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sprint/workflow_manager #1041

Draft
wants to merge 84 commits into
base: release
Choose a base branch
from
Draft

Sprint/workflow_manager #1041

wants to merge 84 commits into from

Commits on Sep 4, 2023

  1. Added some temporary comments in workflow.py.

    @nwieters
    nwieters committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    80d3b6a View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. Started to add docstrings to workflow.py

    @nwieters
    nwieters committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    b63e9a9 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. Added a docstring to esm_plugin_mamager

    @nwieters
    nwieters committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    f9c0659 View commit details
    Browse the repository at this point in the history

  2. Some docstring changes in workflow.py

    @nwieters
    nwieters committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    0c4a962 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Started to improve the workflow documentation.

    @nwieters
    nwieters committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    0452d3b View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Updating the workflow documentation.

    @nwieters
    nwieters committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    63e9cd5 View commit details
    Browse the repository at this point in the history

  2. Added a new general glossary.

    @nwieters
    nwieters committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    14e4b65 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. Added comments to esm_plugin_manager

    @nwieters
    nwieters committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    f0bd385 View commit details
    Browse the repository at this point in the history

  2. Start to refactor workflow.

    @nwieters
    nwieters committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    8c4b753 View commit details
    Browse the repository at this point in the history

  3. Update workflow documentation.

    @nwieters
    nwieters committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    a4a6b34 View commit details
    Browse the repository at this point in the history

  4. Started to refactor workflow.py

    @nwieters
    nwieters committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    e329e14 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Placed the default definition of workflow phases in esm_software.../d… 

    …efauls.yaml .
    @nwieters
    nwieters committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    d7f4565 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2023

  1. Continued to improve documentationfor workflow manager.

    @nwieters
    nwieters committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    34456cb View commit details
    Browse the repository at this point in the history

  2. Add language en in sphinxs conf.py

    @nwieters
    nwieters committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b022c5d View commit details
    Browse the repository at this point in the history

  3. Update src/esm_plugin_manager/esm_plugin_manager.py 

    Co-authored-by: Miguel <[email protected]>
    @nwieters @mandresm
    nwieters and mandresm authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    a2fc1a4 View commit details
    Browse the repository at this point in the history

  4. Update src/esm_plugin_manager/esm_plugin_manager.py 

    Co-authored-by: Miguel <[email protected]>
    @nwieters @mandresm
    nwieters and mandresm authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    fef151d View commit details
    Browse the repository at this point in the history

  5. Update src/esm_runscripts/workflow.py 

    Co-authored-by: Miguel <[email protected]>
    @nwieters @mandresm
    nwieters and mandresm authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    c733a00 View commit details
    Browse the repository at this point in the history

  6. Correcte syntax of docstrings.

    @nwieters
    nwieters committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    506c0a7 View commit details
    Browse the repository at this point in the history

  7. Renamed default_workflow_phases entry in defaults.yaml

    @nwieters
    nwieters committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    03b6e24 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. Renamed workflow method, added property decorator and added test.

    @nwieters
    nwieters committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    b3b5fb1 View commit details
    Browse the repository at this point in the history

  2. Added a temporary workflow to awicm3.

    @nwieters
    nwieters committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    6260278 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Added method to return a list of an attribute for all phases.

    @nwieters
    nwieters committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    de0e598 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Convert functions into methods, moved class variables to be instance … 

    …variables.
    @nwieters
    nwieters committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    11fa96b View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Further developments until order_cluster.

    @nwieters
    nwieters committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    b8d1db0 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. Further changes for workflow manager.

    @nwieters
    nwieters committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    2cd0da6 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Merge branch 'release' into sprint/workflow_manager

    @nwieters
    nwieters committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    98f4a7f View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2023

  1. Added prepend_newrun, skip_cluster, fixed next_submit entries.

    @nwieters
    nwieters committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    273175f View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2023

  1. Reactivated function display_nicely.

    @nwieters
    nwieters committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    3c72864 View commit details
    Browse the repository at this point in the history

  2. Added -i option to esm_runscripts usage

    @nwieters
    nwieters committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    47648ef View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Update documentation on how to inpect workflow.

    @nwieters
    nwieters committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    a0ae28a View commit details
    Browse the repository at this point in the history

  2. Added some flake8 style optimization.

    @nwieters
    nwieters committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    547edb0 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. Update workflow examples.

    @nwieters
    nwieters committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    6ab3f12 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'release' into sprint/workflow_manager_documentation

    @nwieters
    nwieters committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    43c40ca View commit details
    Browse the repository at this point in the history

  3. Adding another example for adding user phases to workflow.

    @nwieters
    nwieters committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    869cc0a View commit details
    Browse the repository at this point in the history

  4. Adapted the output for inspect (-i) workflow.

    @nwieters
    nwieters committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    465d8d3 View commit details
    Browse the repository at this point in the history

  5. Fix multiple phases in one cluster, fix batch_or_shell to be set by e… 

    …sm_tools.
    @nwieters
    nwieters committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    6aee395 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2023

  1. Added some further comments and added cluster info in display_workflo… 

    …w for inspect argument.
    @nwieters
    nwieters committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    138b3c2 View commit details
    Browse the repository at this point in the history

  2. Added section defaults of new workflow phases.

    @nwieters
    nwieters committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    92ea168 View commit details
    Browse the repository at this point in the history

  3. Add new workflow attribute in defaults.yaml.

    @nwieters
    nwieters committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    fcf5a41 View commit details
    Browse the repository at this point in the history

  4. Reactivated function complete_clusters.

    @nwieters
    nwieters committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    5a3bfb8 View commit details
    Browse the repository at this point in the history

  5. Some changes for processing phases in awicm3.

    @nwieters
    nwieters committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    e6bada5 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. Update src/esm_runscripts/inspect.py 

    Co-authored-by: Miguel <[email protected]>
    @nwieters @mandresm
    nwieters and mandresm authored Nov 20, 2023
    Configuration menu
    Copy the full SHA
    ccafa8b View commit details
    Browse the repository at this point in the history

  2. Merged default and user phase class, made phase a subclass of dict.

    @nwieters
    nwieters committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    e107d99 View commit details
    Browse the repository at this point in the history

  3. Fixed some of the workflow tests.

    @nwieters
    nwieters committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    ba204e6 View commit details
    Browse the repository at this point in the history

  4. Added review suggestions.

    @nwieters
    nwieters committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    6174fa3 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Changed initialization of workflow and phases.

    @nwieters
    nwieters committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    93b49b5 View commit details
    Browse the repository at this point in the history

  2. Changed initial config of default workflow phases.

    @nwieters
    nwieters committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    afab16f View commit details
    Browse the repository at this point in the history

  3. Bugfix in error message.

    @nwieters
    nwieters committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    ad0e620 View commit details
    Browse the repository at this point in the history

  4. Make next_run_triggered_by a keyword of phase (trigger_next_run) not … 

    …workflow.
    @nwieters
    nwieters committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    91f1af6 View commit details
    Browse the repository at this point in the history

  5. Bugfix if no phase trigger_next_run.

    @nwieters
    nwieters committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    aed2376 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2023

  1. Resolved function complete_cluster into other functions, fix next_sub… 

    …mit for clusters.
    @nwieters
    nwieters committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    40c9190 View commit details
    Browse the repository at this point in the history

  2. Removed obsolete functions.

    @nwieters
    nwieters committed Nov 23, 2023
    Configuration menu
    Copy the full SHA
    1a8d121 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2023

  1. (Re)moved redundant code, renamed function for collecting phases.

    @nwieters
    nwieters committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    916a345 View commit details
    Browse the repository at this point in the history

  2. Removed the possibility to set workflow keywords by user.

    @nwieters
    nwieters committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    c264f1a View commit details
    Browse the repository at this point in the history

  3. Changed next_run_triggered_by into phase keyworkd.

    @nwieters
    nwieters committed Nov 24, 2023
    Configuration menu
    Copy the full SHA
    7395a9b View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Worked on workflow tests, and other minor changes to workflow.

    @nwieters
    nwieters committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    cf9d0e3 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Fixed a bug in order_phases_and_clusters

    @nwieters
    nwieters committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    f86202d View commit details
    Browse the repository at this point in the history

  2. Added workflow tests for test examples (in documentation).

    @nwieters
    nwieters committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    59590ed View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Fixed test example_2, add preprocessing phase with adding newrun.

    @nwieters
    nwieters committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    ff7145b View commit details
    Browse the repository at this point in the history

  2. Added test for workflow example 5.

    @nwieters
    nwieters committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    c784380 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Added function to cluster phases after collecting them.

    @nwieters
    nwieters committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    6aa7684 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Changes to pass all example tests.

    @nwieters
    nwieters committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    58e3729 View commit details
    Browse the repository at this point in the history

  2. Code optimizations

    @nwieters
    nwieters committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    ecf6bbb View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. Removed unused function.

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    97d5d28 View commit details
    Browse the repository at this point in the history

  2. Finished workflow tests.

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    04d8ff0 View commit details
    Browse the repository at this point in the history

  3. Changes in default workflow definition.

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    16187d9 View commit details
    Browse the repository at this point in the history

  4. Revive init_workflow funtion.

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    f26a115 View commit details
    Browse the repository at this point in the history

  5. Added my email to setup.py

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    08d6976 View commit details
    Browse the repository at this point in the history

  6. Added comments and docstrings.

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    7e1dd6b View commit details
    Browse the repository at this point in the history

  7. Merge branch 'release' into sprint/workflow_manager

    @nwieters
    nwieters committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    f57214d View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Some more code improvements.

    @nwieters
    nwieters committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d1f5a21 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Small changes in some comments.

    @nwieters
    nwieters committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    21fec58 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Remove calc of nproc in workflow.py.

    @nwieters
    nwieters committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    67d08f8 View commit details
    Browse the repository at this point in the history

  2. Remove keyword run_on_queue (not used) and give error by missing run_… 

    …after/run_before.
    @nwieters
    nwieters committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    582caa1 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Adapted log-file name and append mode.

    @nwieters
    nwieters committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    698a523 View commit details
    Browse the repository at this point in the history

  2. Removed run_on_queue from default phases.

    @nwieters
    nwieters committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    bc6c082 View commit details
    Browse the repository at this point in the history

  3. Removed run_on_queue.

    @nwieters
    nwieters committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    05a03ab View commit details
    Browse the repository at this point in the history

  4. Added START and END statement to logfile of recipe steps.

    @nwieters
    nwieters committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    8f75e6f View commit details
    Browse the repository at this point in the history

  5. Added comments.

    @nwieters
    nwieters committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    4bfff6a View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Added comments.

    @nwieters
    nwieters committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    18fd44a View commit details
    Browse the repository at this point in the history

  2. Change comment

    @nwieters
    nwieters committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    cdd7c20 View commit details
    Browse the repository at this point in the history

  3. Adde jobid to logfiles.

    @nwieters
    nwieters committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    638a1ea View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Comment out unused functions.

    @nwieters
    nwieters committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    ff0724c View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Merge pull request #1171 from esm-tools/sprint/workflow_manager_docum… 

    …entation

Sprint/workflow manager documentation
    @nwieters
    nwieters authored Apr 30, 2024
    Configuration menu
    Copy the full SHA
    e49eac8 View commit details
    Browse the repository at this point in the history