Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected __module__ attribute for NamedTupleType.__new__ method: namedtuple_{typename} #127187

Open
XuehaiPan opened this issue Nov 23, 2024 · 3 comments · May be fixed by #127188
Open

Unexpected __module__ attribute for NamedTupleType.__new__ method: namedtuple_{typename} #127187

XuehaiPan opened this issue Nov 23, 2024 · 3 comments · May be fixed by #127188
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@XuehaiPan
Copy link
Contributor

XuehaiPan commented Nov 23, 2024

Bug report

Bug description:

The __module__ attribute for NamedTupleType.__new__ method is namedtuple_{typename}. It does not exist. The __module__ attribute for methods should respect the module argument passed to collections.namedtuple.

In [1]: import collections

In [2]: MyTuple = collections.namedtuple('MyTuple', ['x', 'y', 'z'])

In [3]: MyTuple.__module__
Out[3]: '__main__'

In [4]: MyTuple.__new__.__module__
Out[4]: 'namedtuple_MyTuple'

In [5]: MyTuple._make.__module__
Out[5]: 'collections'

In [6]: import typing

In [7]: class MyAnotherTuple(typing.NamedTuple):
   ...:     a: int
   ...:     b: float
   ...:     

In [8]: MyAnotherTuple.__module__
Out[8]: '__main__'

In [9]: MyAnotherTuple.__new__.__module__
Out[9]: 'namedtuple_MyAnotherTuple'

In [10]: MyAnotherTuple._make.__module__
Out[10]: 'collections'

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

@XuehaiPan XuehaiPan added the type-bug An unexpected behavior, bug, or error label Nov 23, 2024
@rhettinger
Copy link
Contributor

rhettinger commented Nov 23, 2024

Is there any reason users would care about this? Is there any real world impact?

The current result seems reasonable to me. The __new__ method was created virtually so it is fair to report it as `'namedtuple_MyTuple'. It has been this way for a very long time and has not been a source of problems.

@rhettinger rhettinger self-assigned this Nov 23, 2024
@XuehaiPan
Copy link
Contributor Author

XuehaiPan commented Nov 23, 2024

Is there any reason users would care about this? Is there any real world impact?

I'm trying to improve namedtuple support in PyTorch Dynamo, an ML compiler backend with a lightweight Python interpreter. Dynamo traces the Python frames and generates performat inlined functions via traced opcodes. The __module__ attribute is used to reconstruct the Python function during codegen phase.

The codegen tries to import the __name__ attribute during creating the __new__ function for namedtuple

https://github.com/pytorch/pytorch/blob/bae951030752f2d7f06c15af0c59e54026d851e9/torch/_dynamo/symbolic_convert.py#L3290-L3299

    def get_globals_source_and_value(self, name):
        if "__name__" in self.f_globals:
            module_name = self.f_globals["__name__"]
            module_source = self.import_source(module_name)
            if "torch_package" in module_name:
                fglobals_value = torch.package.package_importer._package_imported_modules[module_name]  # type: ignore[assignment]
            else:
                fglobals_value = importlib.import_module(module_name)  # type: ignore[assignment]
            fglobals_vt = VariableTracker.build(self, fglobals_value, module_source)
            global_source = AttrSource(module_source, name)

@picnixz picnixz added the stdlib Python modules in the Lib dir label Nov 23, 2024
@rhettinger
Copy link
Contributor

rhettinger commented Nov 23, 2024

For PyTorch is this a show stopper or is a workaround available?

How would MyTuple.__new__.__module__ returning __main__ actually be useful for reconstructing a __new__ method? That isn't clear to me from your post. That attribute doesn't affect functionality. Technically, a namedtuple doesn't have to live in a module at all. It can be execed dynamically, created and used within a function's locals, stored in a class definination, or stored in a data structure rather than a module's globals. In these circumstances, returning __main__ isn't really correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants