Skip to content

Commit

Permalink
add a draft blog, fix load worktree submited by daemon
Browse files Browse the repository at this point in the history
  • Loading branch information
@superstar54
superstar54 committed Nov 24, 2023
1 parent bb7ccec commit fca7717
Show file tree
Hide file tree
Showing 5 changed files with 805 additions and 5 deletions.
3 changes: 3 additions & 0 deletions aiida_worktree/engine/worktree.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,7 @@ def run_nodes(self, names):
elif node["metadata"]["node_type"] in ["worktree"]:
# process = run_get_node(executor, *args, **kwargs)
from aiida_worktree.utils import merge_properties
from aiida.orm.utils.serialize import serialize

print("node type: worktree.")
wt = self.run_executor(executor, args, kwargs, var_args, var_kwargs)
Expand All @@ -641,6 +642,8 @@ def run_nodes(self, names):
all = {"nt": ntdata, "metadata": {"call_link_label": name}}
print("submit worktree: ")
process = self.submit(self.__class__, **all)
# save the ntdata to the process extras, so that we can load the worktree
process.base.extras.set("nt", serialize(ntdata))
node["process"] = process
# self.ctx.nodes[name]["group_outputs"] = executor.group_outputs
self.ctx.nodes[name]["state"] = "RUNNING"
Expand Down
260 changes: 260 additions & 0 deletions docs/source/blog.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,260 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "22d177dc-6cfb-4de2-9509-f1eb45e10cf2",
"metadata": {},
"source": [
"# Blog: Unlocking Flexibility with AiiDA-WorkTree"
]
},
{
"cell_type": "markdown",
"id": "58696c91",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"In the realm of AiiDA, workflows have been traditionally orchestrated using two chief components: `workfunction` and `WorkChain`. Both these components come with their distinct attributes and limitations. The `workfunction` shines in simplicity but lacks automatic checkpointing, a crucial feature for long-running calculations. Conversely, `WorkChain` embodies resilience with automatic checkpointing but demands a more intricate implementation and lacks flexibility.\n",
"\n",
"\n",
"Suppose we want to calculate ```(x + y) * z ``` in two steps. First, add `x` and `y`, then multiply the result with `z`.\n",
"\n",
"#### A example workchain\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ad4cf60d",
"metadata": {},
"outputs": [],
"source": [
"%load_ext aiida\n",
"from aiida import load_profile\n",
"load_profile()\n",
"# -*- coding: utf-8 -*-\n",
"from aiida.engine import WorkChain, calcfunction\n",
"from aiida.orm import Int\n",
"\n",
"\n",
"@calcfunction\n",
"def add(x, y):\n",
" return Int(x + y)\n",
"\n",
"@calcfunction\n",
"def multiply(x, y):\n",
" return Int(x * y)\n",
"\n",
"class AddAndMultiplyWorkChain(WorkChain):\n",
"\n",
" @classmethod\n",
" def define(cls, spec):\n",
" super().define(spec)\n",
" spec.input('x')\n",
" spec.input('y')\n",
" spec.input('z')\n",
" spec.outline(\n",
" cls.add,\n",
" cls.multiply,\n",
" cls.results,\n",
" )\n",
" spec.output('result')\n",
"\n",
" def add(self):\n",
" self.ctx.sum = add(self.inputs.x, self.inputs.y)\n",
"\n",
" def multiply(self):\n",
" self.ctx.product = multiply(self.ctx.sum, self.inputs.z)\n",
"\n",
" def results(self):\n",
" self.out('result', self.ctx.product)\n"
]
},
{
"cell_type": "markdown",
"id": "15f6f7c5",
"metadata": {},
"source": [
"\n",
"### **Enter AiiDA-WorkTree: The Game-Changer**\n",
"\n",
"Addressing the gaps left by its predecessors, AiiDA-WorkTree introduces a new component: `WorkTree`. This feature simplifies implementation and embraces automatic checkpointing, all while maintaining a high degree of flexibility suitable for designing sophisticated workflows.\n",
"\n",
"In AiiDA, one uses the `calcfunction` to do the calculation and generate new data. AiiDA-WorkTree goes one step further by transforming a `calcfunction` to a `Node`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "c6b83fb5",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Profile<uuid='4a9501fb20364cecb99920cadf27b238' name='xing'>"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from aiida_worktree import node\n",
"from aiida.engine import calcfunction\n",
"\n",
"# define add node\n",
"@node()\n",
"@calcfunction\n",
"def add(x, y):\n",
" return x + y\n",
"\n",
"# define multiply node\n",
"@node()\n",
"@calcfunction\n",
"def multiply(x, y):\n",
" return x*y\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "95855f72",
"metadata": {},
"source": [
"#### Create the workflow\n",
"Three steps:\n",
"\n",
"- create a empty `WorkTree`\n",
"- add nodes: `add` and `multiply`.\n",
"- link the output of the `add` node to one of the `x` input of the `multiply` node."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83f2505a",
"metadata": {},
"outputs": [],
"source": [
"\n",
"from aiida_worktree import WorkTree\n",
"from aiida.orm import Int\n",
"x = Int(2.0)\n",
"y = Int(3.0)\n",
"z = Int(4.0)\n",
"\n",
"wt = WorkTree(\"first_workflow\")\n",
"wt.nodes.new(add, name=\"add\", x=x, y=y)\n",
"wt.nodes.new(multiply, name=\"multiply\", y=z)\n",
"wt.links.new(wt.nodes[\"add\"].outputs[0], wt.nodes[\"multiply\"].inputs[\"x\"])\n",
"\n",
"wt.submit(wait=True)"
]
},
{
"cell_type": "markdown",
"id": "30719f9a",
"metadata": {},
"source": [
"\n",
"### **Journey Through a Workflow: A Train Analogy**\n",
"\n",
"Imagine planning a train journey. Traditionally, using a WorkChain resembles creating a new train line for every unique journey. Traveling from Bern to Zurich? That's one line (ICE10). Heading from Zurich to Milano? That's another (ICE11). Combining journeys, say from Bern to Milano, would entail the creation of yet another train line (ICE12), despite the redundancy.\n",
"\n",
"However, a more optimized approach exists. A WorkTree allows for a journey where one could travel from Bern to Zurich on ICE10 and then switch to ICE11 from Zurich to Milano. This method eliminates the need for creating a superfluous train line, optimizing resources and improving workflow efficiency.\n",
"\n",
"### **Unpacking the Benefits of WorkTree**\n",
"\n",
"#### **Flexibility in Inputs:**\n",
"\n",
"WorkChains typically require the aggregation of all input parameters at the commencement of the process. Using our train analogy, it is like boarding all passengers in Bern, irrespective of their actual starting point. AiiDA-WorkTree, however, allows for a more nuanced approach, enabling inputs to join at different stages of the workflow.\n",
"\n",
"#### **User-Developer Distinction:**\n",
"\n",
"The advent of WorkTree amplifies the distinction between the AiiDA users and developers. It allows developers to focus on creating foundational WorkChains, likened to establishing essential train routes. Users, on the other hand, can effortlessly utilize and connect these foundational elements to tailor workflows that cater to their specific needs, without delving deep into development.\n",
"\n",
"Certainly! I'll add a new section to your blog that discusses guidelines on when to use WorkTree or WorkChain, considering the nuances of workflow construction and modification needs.\n",
"\n",
"\n",
"## **Choosing the Right Tool: WorkTree or WorkChain?**\n",
"\n",
"In the world of AiiDA workflows, making an informed choice between WorkTree and WorkChain is pivotal. The decision hinges on the nature of your workflow and specific requirements concerning data provenance and workflow modification.\n",
"\n",
"### **When to Opt for WorkTree?**\n",
"\n",
"WorkTree emerges as a strong candidate when the workflow primarily involves linking outputs of one AiiDA process to the inputs of another, irrespective of the complexity and number of links. WorkTree facilitates this with an emphasis on maintaining comprehensive data provenance.\n",
"\n",
"- **Linkage-Centric Workflows**: If your workflow is a symphony of interconnected processes where outputs and inputs flow seamlessly, WorkTree is your conductor, orchestrating these links with finesse.\n",
"\n",
"- **Preserving Data Provenance**: With WorkTree, each data transformation is meticulously tracked, ensuring a rich and reliable record of data provenance.\n",
"\n",
"### **When is WorkChain the Preferable Choice?**\n",
"\n",
"WorkChain should be your go-to option when your workflow demands numerous slight modifications to the outputs before they metamorphose into inputs for subsequent processes.\n",
"\n",
"- **Modification-Heavy Workflows**: In scenarios where outputs undergo substantial tweaking and adjustments, WorkChain simplifies this by allowing easy modifications through context variables, acting as a flexible black box.\n",
"\n",
"- **Complex Flow Control**: WorkChain shows its mettle when dealing with intricate flow controls. If your workflow dances to the tunes of multiple while loops and conditional statements, WorkChain choreographs this complexity with grace, even if it makes the workflow somewhat elongated and intricate.\n",
"\n",
"### **Striking a Balance**\n",
"\n",
"Choosing between WorkTree and WorkChain is not a binary decision but rather a balancing act. While WorkTree shines in straightforward, linkage-centric workflows with a focus on data provenance, WorkChain takes the stage when the spotlight is on output modifications and complex flow controls. Evaluating the specific needs of your workflow and the intrinsic capabilities of each tool will guide you towards making a choice that resonates with efficiency and effectiveness.\n",
"\n",
"### **Conclusion: The AiiDA-WorkTree Vision**\n",
"\n",
"AiiDA-WorkTree does not seek to eclipse the existing workflow components like WorkChain, calcfunction, or calcjob. Instead, it envisions enriching the workflow ecosystem, enabling users to craft flexible, resilient, and user-specific workflows with ease. By doing so, AiiDA-WorkTree heralds a new era of workflow management, fostering creativity, efficiency, and precision in the AiiDA community."
]
},
{
"cell_type": "markdown",
"id": "a0779e9d",
"metadata": {},
"source": [
"## What's Next\n",
"\n",
"| | |\n",
"|---------------|----------------------------------------------------|\n",
"| [Concepts](concept/index.rst) | A brief introduction of WorkTree’s main concepts. |\n",
"| [Tutorials](tutorial/index.rst) | Real-world examples in computational materials science and more. |\n",
"| [HowTo](howto/index.rst) | Advanced topics and tips, e.g flow control using `if`, `for`, `while` and `context`. |"
]
},
{
"cell_type": "markdown",
"id": "6ff140c8",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.4 ('scinode')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
},
"vscode": {
"interpreter": {
"hash": "2f450c1ff08798c4974437dd057310afef0de414c25d1fd960ad375311c3f6ff"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
1 change: 1 addition & 0 deletions docs/source/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Here is a detailed comparison between the ``WorkTree`` with two AiiDA built-in w
installation
tutorial/index
howto/index
blog
concept/index


Expand Down
Loading

0 comments on commit fca7717

Please sign in to comment.