AiiDA Usability feedback

[mbercx]

New users to AiiDA often complain that the learning curve is quite steep. We should think about how to lower the barrier for new users to get started using AiiDA, and generally improve the usability of the code.

In this discussion I want to ask AiiDA users to give ideas/feedback on AiiDA's user experience, so we can identify the biggest hurdles and make life easier for our users. To kick things off, I'll give a few examples of my experience when using AiiDA and what I think can be improved.

Improving docstrings of constructors and methods

A lot of the signatures of the constructors and methods of AiiDA classes rely on *args and **kwds and often do not have complete docstrings. Take for example the constructor of StructureData:

Init signature: StructureData(*args, **kwds)
Docstring:     
This class contains the information about a given structure, i.e. a
collection of sites together with a cell, the
boundary conditions (whether they are periodic or not) and other
related useful information.
Init docstring: Construct a new instance, setting the ``source`` attribute if provided as a keyword argument.
File:           ~/.virtualenvs/aiida-hp/lib/python3.8/site-packages/aiida/orm/nodes/data/structure.py
Type:           AbstractNodeMeta
Subclasses: 

This makes it difficult for a new user to understand how to use the AiiDA classes, without having to consult the documentation (which can sometimes be tricky to find and is often not complete). I think we should:

• Try to avoid relying on *args and **kwds too much and explicitly list the input arguments. I understand this may have downsides with regard to maintainability, but on the other hand it would also make it easier to use the constructors/methods in development with tab-completion etc.

• Make an effort to try and list all input parameters in the docstring. As another example here, I noticed recently that in the constructor docstring of the InstalledCode class, only the computer and filepath_executable input arguments are listed:

  Init signature: InstalledCode(*args, **kwds)
  Docstring:      Data plugin representing an executable code on a remote computer.
  Init docstring:
  Construct a new instance.
  
  :param computer: The remote computer on which the executable is located.
  :param filepath_executable: The absolute filepath of the executable on the remote computer.
  File:           ~/.virtualenvs/aiida-hp/lib/python3.8/site-packages/aiida/orm/nodes/data/code/installed.py
  Type:           AbstractNodeMeta
  Subclasses:     ContainerizedCode

  So a new user would not be immediately aware of other important inputs from the parent constructor (prepend_text, default_calc_job_plugin, ...)

Improve the process of setting up computers/codes

One of the first things new users of AiiDA have to do is set up a computer and code. However, this process doesn't always go very smoothly. Some well-known hurdles are:

• Daunting number of (sometimes very use-case specific) inputs asked by the interactive prompts when setting up or configuring the computer (also see Simplify the interface of verdi computer setup #4981). This might be solved with some better documentation (I wasn't aware of the --non-interactive option when using YAML files to set defaults for a long time, for example) or a tutorial on setting up codes/computers (as discussed during a recent tutorial feedback meeting). However, as discussed in Simplify the interface of verdi computer setup #4981 we could also consider only requesting essential information and setting defaults for most settings/configuration.
• Computers that are set up but not yet configured are not shown by verdi computer list (Make verdi computer list show also unconfigured machines #3355). I.e. the computer is still hidden at this point, and can be seen when adding the -a, --all option, but perhaps we should not hide non-configured computers by default.
• Any information provided during the setup step of the computer/code can no longer be changed afterwards. This means that any mistake (e.g. a typo in the binary path, incorrectly specified default memory, ...) can only be corrected by deleting the faulty computer/code and making a new one. This can be made less painful by using YAML files, but new users are often not aware of this approach. Although it technically doesn't preserve the provenance, I wonder if we can't allow some flexibility here to make life easier for the users and allow changes to e.g. the prepend_text after the computer/code has already been set up.

[superstar54]

Support adding labels for the child processes.

Take PwBandsWorkChain as an example. Here is the output of the process status (from this demo: https://aiida-qe-demo.readthedocs.io/en/latest/2_bands_workflow.html)

%verdi process status {result.node.pk}

PwBandsWorkChain<104> Finished [0] [7:results]
    ├── PwRelaxWorkChain<106> Finished [0] [3:results]
    │   ├── PwBaseWorkChain<109> Finished [0] [4:results]
    │   │   ├── create_kpoints_from_distance<110> Finished [0]
    │   │   └── PwCalculation<114> Finished [0]
    │   └── PwBaseWorkChain<123> Finished [0] [4:results]
    │       ├── create_kpoints_from_distance<124> Finished [0]
    │       └── PwCalculation<128> Finished [0]
    ├── seekpath_structure_analysis<135> Finished [0]
    ├── PwBaseWorkChain<142> Finished [0] [4:results]
    │   ├── create_kpoints_from_distance<143> Finished [0]
    │   └── PwCalculation<147> Finished [0]
    └── PwBaseWorkChain<155> Finished [0] [4:results]
        └── PwCalculation<158> Finished [0]

There are too many PwBaseWorkChain and PwCalculation, and one can not distinguish them easily.
It would be a great help to show the following:

PwBandsWorkChain<104> Finished [0] [7:results]
├── PwRelaxWorkChain<106> Finished [0] [3:results]
│ ├── relax1 <109> Finished [0] [4:results]
│ │ ├── create_kpoints_from_distance<110> Finished [0]
│ │ └── PwCalculation<114> Finished [0]
│ └── realx2 <123> Finished [0] [4:results]
│ ├── create_kpoints_from_distance<124> Finished [0]
│ └── PwCalculation<128> Finished [0]
├── seekpath_structure_analysis<135> Finished [0]
├── pw_scf <142> Finished [0] [4:results]
│ ├── create_kpoints_from_distance<143> Finished [0]
│ └── PwCalculation<147> Finished [0]
└── pw_nscf <155> Finished [0] [4:results]
└── PwCalculation<158> Finished [0]

[zhubonan]

For child process launch the parents can already customise their labels with metadata.label as well metadata.call_link_label. I am not sure if the QE workchain stack uses them.

That is a good idea, we just need allow the link name to be printed with verdi process status?

[Crivella]

For the discussion on the features to improve the usability of the QueryBuilder, my idea is to implement a "Search bar" like tool where a human readable string is converted in a simple query (initially just the function, than it could be converted in a nice WebUI with autocompletion/suggestions).

EG: In a simpler database structure, like a movie database i would be able to run a query of the type
title:"That movie" author:"whoever" release:">1900-01-01"

In the AiiDA case where the structure is more treelike, we would have to discuss how to implement this, mostly how many feature the query string would have, but i would keep it as simple as possible, something like:
CalcJob:"pw" Group:"some label or part of label"
would return all pw calcjobs belonging to groups matching that label (I used intentionally only pw, because i think the string should be able to work in the most generic case matching any plugin with a pw calcjob entrypoint and than could be restricted using CalcJob:"quantumespresso.pw"

The core of this feature would be the parser that transform the string into a query, but for userfriendliness, having an actual searchbar with an autocomplete/suggest feature would make this usable by someone who has never seen AiiDA or a query

[Crivella]

PS: maybe looking into if there are query systems for services that already deal with Tree like data (family tree?) to get an idea of what feature should be included in the search string should be the first step so we already have a design in mind