Skip to content

Commit

Permalink
Docs: Move limitations warning to top of quick install
Browse files Browse the repository at this point in the history
The fact that the quick installation guide can lead to a profile that
has certain limitations should be the first thing to be read.
  • Loading branch information
sphuber committed Jul 22, 2024
1 parent 96d9fbd commit d73731f
Show file tree
Hide file tree
Showing 2 changed files with 58 additions and 42 deletions.
21 changes: 14 additions & 7 deletions docs/source/installation/guide_complete.rst
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ Common options

The exact options available for the ``verdi profile setup`` command depend on the selected storage plugin, but there are a number of common options and functionality:

* ``--profile``: The name of the profile.
* ``--profile-name``: The name of the profile.
* ``--set-as-default``: Whether the new profile should be defined as the new default.
* ``--email``: Email for the default user that is created.
* ``--first-name``: First name for the default user that is created.
Expand All @@ -276,6 +276,10 @@ The exact options available for the ``verdi profile setup`` command depend on th
``core.sqlite_dos``
-------------------

.. tip::

The ``verdi presto`` command provides a fully automated way to set up a profile with the ``core.sqlite_dos`` storage plugin if no configuration is required.

This storage plugin uses `SQLite <https://sqlite.org/>`_ and the `disk-objectstore <https://disk-objectstore.readthedocs.io/en/latest/>`_ to store data.
The ``disk-objectstore`` is a Python package that is automatically installed as a dependency when installing ``aiida-core``, which was covered in the :ref:`Python package installation section <installation:guide-complete:python-package>`.
The installation instructions for SQLite depend on your system; please visit the `SQLite website <https://www.sqlite.org/download.html>`_ for details.
Expand All @@ -296,20 +300,23 @@ The options specific to the ``core.sqlite_dos`` storage plugin are:
``core.psql_dos``
-----------------

This storage plugin uses `PostgreSQL <https://www.postgresql.org/>`_ and the `disk-objectstore <https://disk-objectstore.readthedocs.io/en/latest/>`_ to store data.
The ``disk-objectstore`` is a Python package that is automatically installed as a dependency when installing ``aiida-core``, which was covered in the :ref:`Python package installation section <installation:guide-complete:python-package>`.
The storage plugin can connect to a PostgreSQL instance running on the localhost or on a server that can be reached over the internet.
Instructions for installing PostgreSQL is beyond the scope of this guide.

.. tip::

The creation of the PostgreSQL user and database as explained below is implemented in an automated way in the ``verdi presto`` command.
Instead of performing the steps below manually and running ``verdi profile setup core.psql_dos`` manually, it is possible to run:
Instead of performing the steps below manually and running ``verdi profile setup core.psql_dos``, it is possible to run:

.. code-block::
verdi presto --use-postgres
The ``verdi presto`` command also automatically tries to configure RabbitMQ as the broker if it is running locally.
Therefore, if the command succeeds in connecting to both PostgreSQL and RabbitMQ, ``verdi presto --use-postgres`` provides a fully automated way to create a profile suitable for production workloads.

This storage plugin uses `PostgreSQL <https://www.postgresql.org/>`_ and the `disk-objectstore <https://disk-objectstore.readthedocs.io/en/latest/>`_ to store data.
The ``disk-objectstore`` is a Python package that is automatically installed as a dependency when installing ``aiida-core``, which was covered in the :ref:`Python package installation section <installation:guide-complete:python-package>`.
The storage plugin can connect to a PostgreSQL instance running on the localhost or on a server that can be reached over the internet.
Instructions for installing PostgreSQL is beyond the scope of this guide.

Before creating a profile, a database (and optionally a custom database user) has to be created.
First, connect to PostgreSQL using ``psql``, the `native command line client for PostgreSQL <https://www.postgresql.org/docs/current/app-psql.html>`_:

Expand Down
79 changes: 44 additions & 35 deletions docs/source/installation/guide_quick.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
Quick installation guide
========================

.. warning::

Not all AiiDA functionality is supported by the quick installation.
Please refer to the :ref:`section below <installation:guide-quick:limitations>` for more information and see the :ref:`complete installation guide <installation:guide-complete>` for instructions to set up a feature-complete and performant installation.


First, install the ``aiida-core`` Python package:

.. code-block:: console
Expand Down Expand Up @@ -35,64 +41,67 @@ If none of the lines show a red cross, indicating a problem, the installation wa

If you encountered any issues, please refer to the :ref:`troubleshooting section <installation:troubleshooting>`.

.. warning::

Not all AiiDA functionality is supported by the quick installation.
Please refer to the :ref:`section below <installation:guide-quick:limitations>` for more information.


.. _installation:guide-quick:limitations:

Quick install limitations
=========================

By default, ``verdi presto`` creates a profile that uses SQLite instead of PostgreSQL and does not use the RabbitMQ message broker.
The table below gives a quick overview of the functionality that is not supported in those cases:
A setup that is ideal for production work requires the PostgreSQL and RabbitMQ services.
By default, ``verdi presto`` creates a profile that allows running AiiDA without these:

+-----------------------------------------+------------------------------------------------------------------------+
| No RabbitMQ | SQLite instead of PostgreSQL |
+=========================================+========================================================================+
| Cannot run the daemon | Not suitable for high-throughput workloads |
+-----------------------------------------+------------------------------------------------------------------------+
| Cannot submit processes to the daemon\* | No support for ``has_key`` and ``contains`` operators in query builder |
+-----------------------------------------+------------------------------------------------------------------------+
| Cannot play, pause, kill processes | No support for ``QueryBuilder.get_creation_statistics`` |
+-----------------------------------------+------------------------------------------------------------------------+
* **Database**: The PostgreSQL database that is used to store queryable data, is replaced by SQLite.
* **Broker**: The RabbitMQ message broker that allows communication with and between processes is disabled.

\* Calculations can still be run on remote computers
The following matrix shows the possible combinations of the database and broker options and their use cases:

.. note::
To enable the RabbitMQ broker for an existing profile, :ref:`install RabbitMQ <installation:guide-complete:rabbitmq>` and then run ``verdi profile configure-rabbitmq``.
+----------------------+----------------------------------------------------+-------------------------------------------------------------+
| | **SQLite database** | **PostgreSQL database** |
+======================+====================================================+=============================================================+
| **No broker** | Quick start with AiiDA | [*not really relevant for any usecase*] |
+----------------------+----------------------------------------------------+-------------------------------------------------------------+
| **RabbitMQ** | Production (low-throughput; beta, has limitations) | Production (maximum performance, ideal for high-throughput) |
+----------------------+----------------------------------------------------+-------------------------------------------------------------+

The sections below provide details on the use of the PostgreSQL and RabbitMQ services and the limitations when running AiiDA without them.

.. _installation:guide-quick:limitations:rabbitmq:

Functionality
-------------
RabbitMQ
--------

Part of AiiDA's functionality requires a `message broker <https://en.wikipedia.org/wiki/Message_broker>`_, with the default implementation using `RabbitMQ <https://www.rabbitmq.com/>`_.
The message broker is used to allow communication with the :ref:`daemon <topics:daemon>`.
Since RabbitMQ is a separate service and is not always trivial to install, the quick installation guide sets up a profile that does not require it.
As a result, the daemon cannot be started and processes cannot be submitted to it but can only be run in the current Python interpreter.
The message broker is used to allow communication with processes and the :ref:`daemon <topics:daemon>` as well as between themselves.
Since RabbitMQ is a separate service and is not always trivial to install, the quick installation guide allows setting up a profile that does not require it.
However, as a result the profile:

* is not suitable for high-throughput workloads (a polling-based mechanism is used rather than an event-based one)
* cannot run the daemon (no ``verdi daemon start/stop``) and therefore processes cannot be submitted to the daemon (i.e., one can only use ``run()`` instead of ``submit()`` to launch calculations and workflows)
* cannot play, pause, kill processes

.. note::
The ``verdi presto`` command automatically checks if RabbitMQ is running on the localhost.
If it can successfully connect, it configures the profile with the message broker and therefore the daemon functionality will be available.
If it can successfully connect, it configures the profile with the message broker and therefore the limitations listed above do not apply.

.. tip::
The connection parameters of RabbitMQ can be (re)configured after the profile is set up with ``verdi profile configure-rabbitmq``.
This can be useful when the RabbitMQ setup is different from the default that AiiDA checks for and the automatic configuration of ``verdi presto`` failed.
A profile created by ``verdi presto`` can easily start using RabbitMQ as the broker at a later stage.
Once a RabbitMQ service is available (see :ref:`install RabbitMQ <installation:guide-complete:rabbitmq>` for instruction to install it) and run ``verdi profile configure-rabbitmq`` to configure its use for the profile.

.. _installation:guide-quick:limitations:postgresql:

PostgreSQL
----------

Performance
-----------
AiiDA stores (part of) the data of the provenance graph in a database and the `PostgreSQL <https://www.postgresql.org/>`_ service provides great performance for use-cases that require high-throughput.
Since PostgreSQL is a separate service and is not always trivial to install, the quick installation guide allows setting up a profile that uses the serverless `SQLite <https://www.sqlite.org/>`_ instead.
However, as a result the profile:

The quick installation guide by default creates a profile that uses `SQLite <https://www.sqlite.org/>`_ for the database.
Since SQLite does not require running a service, it is easy to install and use on essentially any system.
However, for certain use cases it is not going to be the most performant solution.
AiiDA also supports `PostgreSQL <https://www.postgresql.org/>`_ which is often going to be more performant compared to SQLite.
* is not suitable for high-throughput workloads (concurrent writes from multiple processes to the database are serialized)
* does not support the ``has_key`` and ``contains`` operators in the ``QueryBuilder``
* does not support the ``get_creation_statistics`` method of the ``QueryBuilder``

.. tip::
If a PostgreSQL service is available, run ``verdi presto --use-postgres`` to set up a profile that uses PostgreSQL instead of SQLite.
The command tries to connect to the service and automatically create a user account and database to use for the new profile.
AiiDA provides defaults that work for most setups where PostgreSQL is installed on the localhost.
Should this fail, the connection parameters can be customized using the ``--postgres-hostname``, ``--postgres-port``, ``--postgres-username``, ``--postgres-password`` options.

Please refer to the :ref:`complete installation guide <installation:guide-complete>` for instructions to set up a feature-complete and performant installation.

0 comments on commit d73731f

Please sign in to comment.