Questions about several "issues" I found with the docstring parser.

[Masara]

I found several "issues" with the docstring parser which I want to ask about and discuss here first, since I don't know if these "issues" are intentional by design or not.

1. Numpydoc: Grouped attributes

Grouped attributes in numpy can't be parsed for attributes, only for parameters. Take for example this code:

def function_with_parameters(grouped_parameter_1, grouped_parameter_2)-> None:
    """
    Function with parameters.

    Dolor sit amet.

    Parameters
    ----------
    grouped_parameter_1, grouped_parameter_2 : int, default=4
        foo: grouped_parameter_1 and grouped_parameter_2
    """

class ClassWithAttributesAndParameters:
    """
    Class with attributes and parameters.

    Dolor sit amet.


    Parameters
    ----------
    grouped_attribute_3, grouped_attribute_4 : int, default=4
        foo: grouped_attribute_3 and grouped_attribute_4

    Attributes
    ----------
    grouped_attribute_1, grouped_attribute_2 : int, default=4
        foo: grouped_attribute_1 and grouped_attribute_2
    """
    grouped_attribute_1, grouped_attribute_2 = 4, 4

    def __init__(self, grouped_attribute_3, grouped_attribute_4) -> None:
        pass

For parameter docstrings (in classes and functions) grouped parameters can be parsed correctly, but not for attributes. Is this intended?

2. Google & Sphinx: Can't parse default values in docstrings

Google and Sphinx describe default values with "defaults to", e.g.:

class ClassWithGoogleDocstring:
    """Lorem ipsum.

    Dolor sit amet.

    Args:
        p (int): foo. Defaults to 1.
    """

    def __init__(self, p) -> None:
        pass

class ClassWithSphinxDocstring:
    """
    Lorem ipsum.

    Dolor sit amet.

    :param p: foo defaults to 1
    :type p: int
    """

    def __init__(self, p) -> None:
        pass

These values can't be parsed with Griffe though.

3. Sphinx attribute types can't be parsed

Parameters defined with param can be parsed normally to get the description and the type. If we use var (also cvar or ivar) though we get only the description, not the type. Example code:

class SphinxClassWithAttributesAndParameters:
    """
    Lorem ipsum.

    Dolor sit amet.

    :param p: foo 
    :type p: int
    :var q: bar
    :type q: int
    """
    q = 1

    def __init__(self, p) -> None:
        pass

4. optional for types in docstrings only works in some cases

If we want to descripe a type in Numpy, Google or Sphinx style, using optional for optional types only works in some cases:

• Numpy: Only works with class attributes.
• Google: Does not work in any case.
• Sphinx: Works with function and class parameters. Attribute types can't be parsed, see 3rd issue above.

For the cases where it actually works, we get an ExprTuple with optional as one of the items.

See this example code:

# ### Numpy ### #
# Here "optional" can be parsed.
class NumpyClassWithOptionalAttribute:
    """
    Lorem ipsum.

    Dolor sit amet.

    Attributes
    ----------
    optional_attr : int, optional
        foo: optional attribute
    """
    optional_attr = None

# Here "optional" can NOT be parsed.
class NumpyClassWithOptionalParameter:
    """
    Lorem ipsum.

    Dolor sit amet.

    Parameters
    ----------
    optional_param : int, optional
    """

    def __init__(self, optional_param) -> None:
        pass

# Here "optional" can NOT be parsed.
def numpy_function_with_optional_parameter(optional_param) -> None:
    """
    Lorem ipsum.

    Dolor sit amet.

    Parameters
    ----------
    optional_param : int, optional
    """

# ### Google ### #
# Here "optional" can NOT be parsed.
class GoogleClassWithOptionalAttribute:
    """Lorem ipsum.

    Dolor sit amet.

    Attributes:
        optional_attr (int, optional): Description...
    """
    optional_attr = None

# Here "optional" can NOT be parsed.
class GoogleClassWithOptionalParameter:
    """Lorem ipsum.

    Dolor sit amet.

    Args:
        optional_param (int, optional): Description...
    """

    def __init__(self, optional_param) -> None:
        pass

# Here "optional" can NOT be parsed.
def google_function_with_optional_parameter(optional_param) -> None:
    """Lorem ipsum.

    Dolor sit amet.

    Args:
        optional_param (int, optional): Description...
    """

# ### Sphinx ### #
# Here "optional" can be parsed.
class SphinxClassWithOptionalParameter:
    """
    Lorem ipsum.

    Dolor sit amet.

    :param optional_param:
    :type optional_param: int, optional
    """

    def __init__(self, optional_param) -> None:
        pass


# Here "optional" can be parsed.
def sphinx_function_with_optional_parameter(optional_param) -> None:
    """
    Lorem ipsum.

    Dolor sit amet.

    :param optional_param: optional type
    :type optional_param: int, optional
    """

[pawamoy]

Hey, thanks a lot for this detailed report!

1. For parameter docstrings (in classes and functions) grouped parameters can be parsed correctly, but not for attributes. Is this intended?

   The Numpydoc style guide is unclear and doesn't explicitly tell attributes can be grouped.

2. p (int): foo. Defaults to 1.
   :param p: foo defaults to 1

   This is arbitrary text and cannot be parsed indeed. It could be written in something else than english, or use different punctuation, spacing, or describe complex values such as some value if something else another value which cannot be translated to Python code.

3. var, cvar, ivar

   Simply not implemented. Someone else contributed the Sphinx parser. PRs welcome! I'm myself not interested in improving it, so it's a super low priority to me 🙂

4. NumpyClassWithOptionalAttribute

   IMO it does not make sense to mark an attribute as optional. Attributes are not optional, they are always there. If they can take the value None, then the type should be int | None or Optional[int], not int, optional. Also, once again the style guide is unclear and does not explicitly say attributes can be marked optional. The fact that it's parsed an a tuple expression is simply a side-effect.

   NumpyClassWithOptionalParameter

   This example does not make sense either IMO, because the docstring says the parameter is optional while it's actually required in the signature. In fact, Griffe doesn't do anything with , optional: it just parses it to remove it from the actual type. Whether it's optional and it has a default value (and which default value), is fetched from the signature. Happy to investigate further with a different example 🙂

   numpy_function_with_optional_parameter

   Same, the parameter is actually not optional based on the signature.

   GoogleClassWithOptionalAttribute

   This time Napoelon's docs do show an attribute documented as "optional", though they only seem to do so because the parameter assigned to it is itself marked as optional (but again, it's actually not optional based on the signature...). So IMO the example is contrived and it still does not make sense to mark an attribute as optional. My reasoning is that nowadays we have proper syntax to say "int or None", so we shouldn't use "optional" which is ambiguous.

   GoogleClassWithOptionalParameter

   Same, the parameter is actually not optional based on the signature.

   google_function_with_optional_parameter

   Same, the parameter is actually not optional based on the signature.

   sphinx

   I see it the other way: here optional is NOT parsed and ends up being eaten by the annotation, making it incorrect: int, optional is a tuple, and optional does not exist in the given scope. What I mean is that the , optional part is english text which shouldn't become part of the annotation, which is Python code. Happy to review a PR removing , optional from the annotation 😄

[Masara]

Thank you for your quick and extensive answer! I'm with you with what you explained and see your points. Regarding the support for attributes in Sphinx and ignoring "optional", we will probably create PRs for Griffe someday in the future, but for our project it's currently also low priority.

Once again thank you for your answer! 😁