pySDC v6

[pancetta]

Since I lost track of the ideas/plans, I'd like to bundle the discussion on going over to version 6.0. What do we want/need to call a new version "6.0"?

[tlunet]

I would suggest finalizing #255 and #399, and additionally :

• proper definition of an abstract datatype interface (on which the abstract problem class, sweeper, etc ... would be based)
• proper dev documentation on how to add a new datatype and problem

[lisawim]

One thing that directly comes into my mind is: Testing all the problem classes to make sure that each problem class does do what it is supposed to do. This is still a lot of work though..

Please correct me if I'm wrong but in pySDC there are methods/functions following at least two different documentation conventions:

def func(n):
    """
    Args:
        n (int): Integer.

    Returns:
        m (int): Another integer
    """
    n = m
    return m

and

def func(n):
    """
    Parameters
    ----------
    n : int
        Integer.

    Returns
    -------
    m : int
        Another integer
    """
    n = m
    return m

In #286 @tlunet suggested following the NumPy Style Python Docstring which is also mentioned in our documentation guidelines. However, the format of the first documentation does not follow these guidelines as I assume? Since this is also an ambitious goal until v6 I suggest instead doing this step by step.

[brownbaerchen]

Do we want major GPU compatibility in the next release? If we touch the base problem class, we can abstract the libraries we need for CPU/GPU at the same time. This does introduce some mess because you need both numpy and cupy for GPU implementations. So I don't know if we want it everywhere.
If yes, I suggest a new module similar to the following:

import numpy as np
import scipy.sparse as sp

class _Libs(type):
    np = np
    sp = sp
    xp = np

class Libs(metaclass=_Libs):
    @staticmethod
    def setup_GPU():
        import cupy as cp
        _Libs.xp = cp

        from cupyx.scipy import sparse as cu_sparse
        _Libs.sp = cu_sparse

Which would give a reasonably clean implementation of having the libraries be "global variables."

[tlunet]

This should probably be included in a reworked generic datatype interface, then ... ?

[brownbaerchen]

We should sit down and decide on a good way to do this at some point. Maybe we can actually combine the datatypes for CPU and GPU if we inherit from pySDC.Libs.xp.ndarray. That would be nice. The current code duplication is sad.

[tlunet]

Indeed ... may be during one session of the next SDC days ? 😇 Will be in 6 months tho ...

[brownbaerchen]

Did someone say pySDC marathon?!?

[tlunet]

Marathon takes one day usually ... not sure that will be enough 😅

[brownbaerchen]

Also for version 6, I suggest to make use of __init__.py files to allow for shorter import paths.

[brownbaerchen]

I have seen a few instances where we needed to copy-paste the update of the preconditioner between iterations for the flex preconditioner, for instance in #455. I believe we should put this in a better framework for v6. For instance, we can have a function update_preconditioner which does whatever based on various status objects. Given that there is some interest in further diagonal preconditioners for things like IMEX, DAE, ..., I think this would be useful not just from a code-cleanliness point of view.

[tlunet]

My guess : should be a call depending on the sweep number (at least, maybe other parameters), and done before the update_node call for every sweeper class ... but not very clear how to do that in practice 😅

[brownbaerchen]

It's not specific to the sweeper, but to the preconditioner. So I think we can safely move that to the controllers. Of course they call a function in the core sweeper, but there should be no need for overloading this in specific sweepers.
I would keep it generic and not pass anything. The sweep number is a parameter of the level status, so it's there anyways.

[tlunet]

Agree

[brownbaerchen]

I added a project for v6 here. Feel free to continue writing in this discussion, but please also add a tile for your suggestion in the project such that we keep track of everything. You can also link parts of this discussion in the project as I have done for everything that has been said so far. This allows people to respond easily to your comments.

[lisawim]

Scrolling through a PR for something where I also saw the several convergence controller classes: Since the convergence controllers does have a different order control_order where they are called, would it make sense to also provide an overview in which order the different convergence controllers would be called? Or something like a small tutorial for the different convergence controllers where the ordering is also mentioned? I would add it to the v6 project, but if you feel this suggestion is not a good one I'll removed it again.

[brownbaerchen]

We have this already in the logs. When you instantiate the controller, along with all other configurations, there is a table of all active convergence controllers including their order and if you added them yourself or if they were added as a dependency. Here is an example:

Active convergence controllers:
    |  # | order | convergence controller
----+----+-------+---------------------------------------------------------------------------------------
 -> |  0 |   -75 | EstimatePolynomialError
--> |  1 |   -50 | AdaptivityPolynomialError
 -> |  2 |    91 | StepSizeSlopeLimiter
--> |  3 |    92 | StepSizeLimiter
--> |  4 |    94 | StopAtNan
--> |  5 |    94 | StopAtMaxRuntime
--> |  6 |    95 | BasicRestartingMPI
 -> |  7 |   100 | SpreadStepSizesBlockwiseMPI
    |  8 |   200 | CheckConvergence

I agree that more documentation could not hurt. Or a rename of the whole thing..

[lisawim]

I know about this logs and I really like that overview! However as you wrote, this log only appears for the active ones, and for some new pySDC users it probably does make sense to provide an overview of all convergence controllers together with their control order to avoid having to look up each individual one. Further, a tutorial about the convergence controllers in general could be a good thing since this is not a project-related topic anymore (?, I assume since the convergence controllers are not in the projects folder)

[brownbaerchen]

That is a good point. There may be unexpected some side effects from the wrong order. We should be able to put a table like that on the website that is generated by CI. That is probably a good idea.
I have been unhappy with some of the decisions in the early development of this module class and I think fixing that is a large-ish task. But I do want to break the API by renaming the convergence controllers. In talks I have called them call-backs. I am happy to get naming suggestions... Let's put it on the list and see how far we get with cleaning this up :D

[lisawim]

Your point of unexpecting side effects does make sense! Sounds good, we put it on the list!

In order to make this point even larger: Already in the last 12 months we implemented so many new things here. There are so many features and we could therefore write more stuff for the tutorial where we also explain these new things. We would have then one more reason to tell people why they should use pySDC (in addition to the reason that pySDC is just ❤️ ): Good documentation with good tutorial. I mean this is already the case and I really like the tutorial. But it is like a math book: You can look up there so many things and if you have questions the math book knows the answer. If you would like to implement something, the math book knows the answer. The tutorial still contains the very basics and in my view we should extend it to provide more details of the framework of pySDC. This is a really big task though and not necessarily related to v6.

[lisawim]

I had an interesting idea that I'd like to share with you and that could also be very interesting for the next version: When using the scipy.integrate.solve_ivp routine solvers use adaptive time-stepping. We can pass the flag dense_output=True and with that it is possible to compute solution values at arbitrary time points which is done via interpolation. I think this could be very interesting for pySDC as well: Since pySDC is an iterative method and we want to compute a solution with small time step size (e.g. a reference solution) it does make sense to use adaptivity. However, this does not guarantee solution values at desired time points. I guess we can use the LagrangeApproximation class from @tlunet's qmat package and realize the interpolation with that. What do you think about this idea? I put it also in the pySDC v6 project anyways, and in case you think this is not a good idea I'll remove it then again.

[tlunet]

There is already a t_eval parameter in scipy.integrate.solve_ivp , to specify the times at which to compute the numerical solution. Isn't it not enough for what you want ?

[lisawim]

Yes, and this is really nice! However, I have a DAE problem (btw index one problems can define in the way that solve_ivp can still be used to solve them) and this has a very steep slope at some point so that the integration step in the scipy routine does fail. I get the message that Required step size is less than spacing between numbers. At least for DAE problems this could be interesting since I don't know DAE routines in Python to do that.

[tlunet]

That Required step size is less than spacing between numbers is weird ... is there not any way to force solve_ivp to do sub-timestepping ?