Adding pyproject.toml #302

drfho · 2024-09-03T18:08:02Z

Ref: https://github.com/idasm-unibe-ch/zms-base/issues/3
Ref: https://github.com/zms-publishing/ZMS/blob/276b695799f94ade7a1a527cef793a0154fc6cb3/pyproject.toml

A file pyproject.toml allows pip to install Python modules in editable-mode. pip-generic e-option will be removed from pip in the future. Actually the presence of that file
276b695
will constantly induce an error on starting Zope,

The error can be reproduced when starting

the UniBE docker-compose after setting backend/zms-core to zms.main/latest and adding the file https://github.com/zms-publishing/ZMS/blob/828cd8a11bc96aa958ccdf4dfd2371d9246d54ec/requirements-unibe.txt
the ZMS Ubuntu Docker image
https://github.com/zms-publishing/ZMS/blob/main/docker/ubuntu/dockerfile
with a ZMS-commit that contains the file pyproject.toml
276b695

2024-09-03 17:30:05 INFO [chameleon.config:40][MainThread] directory cache: /home/zope/venv/instance/zms5/var/cache.
Traceback (most recent call last):
  File "/home/zope/venv/bin/runwsgi", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/serve.py", line 251, in main
    return command.run()
           ^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/serve.py", line 189, in run
    app = self.loadapp(app_spec, name=app_name, relative_to=base,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/serve.py", line 220, in loadapp
    return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 246, in loadapp
    return loadobj(APP, uri, name=name, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    return context.create()
           ^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
    return self.object_type.invoke(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 198, in invoke
    app = context.app_context.create()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 738, in create
    return self.object_type.invoke(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/loadwsgi.py", line 136, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/paste/deploy/util.py", line 61, in fix_call
    val = callable(*args, **kw)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/run.py", line 61, in make_wsgi_app
    starter.prepare()
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/starter.py", line 38, in prepare
    self.startZope()
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/Startup/starter.py", line 94, in startZope
    Zope2.startup_wsgi()
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/__init__.py", line 36, in startup_wsgi
    _startup()
  File "/home/zope/venv/lib/python3.12/site-packages/Zope2/App/startup.py", line 70, in startup
    OFS.Application.import_products()
  File "/home/zope/venv/lib/python3.12/site-packages/OFS/Application.py", line 432, in import_products
    for priority, product_name, index, product_dir in get_products():
                                                      ^^^^^^^^^^^^^^
  File "/home/zope/venv/lib/python3.12/site-packages/OFS/Application.py", line 419, in get_products
    product_names = os.listdir(product_dir)
                    ^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'Products'

pydev-Debugger in Py3-12-venv shows an invalid PATH_PLACEHOLDER

Ref: idasm-unibe-ch/zms-base#3 Ref: https://github.com/zms-publishing/ZMS/blob/276b695799f94ade7a1a527cef793a0154fc6cb3/pyproject.toml

drfho · 2024-09-03T19:51:23Z

With Zope-editable pyprojects.toml creates resolvable Products paths

If Zope standard-release is installed non-editable, there will be unresolvable path artifacts like __editable__.ZMS-5.2.0.finder.__path_hook__ in the Products-path list. This blocks the Zope start.
After re-installing Zope-latest in editable-mode like this:

./pip install -e git+https://github.com/zopefoundation/Zope.git@master#egg=Zope

Ref:

ZMS/docker/ubuntu/dockerfile

Line 27 in 79e2107

    
           RUN pip install -U -e git+https://github.com/zopefoundation/Zope.git@master#egg=Zope

the installation is able to swallow the ZMS/pyprojects.toml file and creates valid Products pathes and Zope will start correctly.

dwt · 2024-09-10T07:21:42Z

Zope fängt jetzt an eine pyproject.toml in allen Repos auszurollen

dwt · 2024-09-10T13:50:02Z

Also wenn ich mir den Code in OFS.Application:get_products() anschaue, dann ist der glaube ich einfach fehlerhaft? Verglichen mit der Python-Packaging Guide hätte ich in etwa so etwas erwartet:

(Pdb) pp list(pkgutil.iter_modules(Products.__path__, Products.__name__ + '.'))
[ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.BTreeFolder2', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.CMFCore', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.ExternalMethod', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.Five', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.GenericSetup', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.LDAPMultiPlugins', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.LDAPUserFolder', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.MailHost', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.OFSP', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.PageTemplates', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.PluggableAuthService', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.PluginIndexes', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.PluginRegistry', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.PythonScripts', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.SQLAlchemyDA', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.Sessions', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.SiteAccess', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.SiteErrorLog', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.StandardCacheManagers', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.TemporaryFolder', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.Transience', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.ZCTextIndex', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.ZCatalog', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.ZODBMountPoint', ispkg=True),
 ModuleInfo(module_finder=FileFinder('/home/zope/venv/lib/python3.12/site-packages/Products'), name='Products.ZSQLMethods', ispkg=True)]

Die spannende Frage: Findet das auch alle Pakete? Weil in

(Pdb) pp Products.__path__
_NamespacePath(['/home/zope/venv/lib/python3.12/site-packages/Products', 'Products', '__editable__.ZMS-5.2.0.finder.__path_hook__'])

Hier fehlt mir aber Products.zms was sich prima importieren lässt, aber hier nicht auftaucht.

dwt · 2024-09-10T14:04:47Z

Dieser Bug in Setuptools scheint relevant zu sein

dwt · 2024-11-19T12:16:03Z

Hier ist ausführlich dokumentiert was sich geändert hat in pip (Man muss sich aber durch einige Links durchfräsen)

dwt · 2024-11-26T10:53:41Z

zopefoundation/Zope#1239 has some reaction from zope devs

dwt · 2024-12-10T09:58:14Z

Zope hat in main die Art, wie sie installierte Produkte lesen, geändert. Das sollte das von uns gesehene Problem lösen: zopefoundation/Zope#1240

Sobald wir verifiziert haben, dass diese Lösung funktioniert und eine Zope Version mit diesem Fix released ist, können wir, denke ich, mergen.

adding pyproject.toml

c6f13ea

Ref: idasm-unibe-ch/zms-base#3 Ref: https://github.com/zms-publishing/ZMS/blob/276b695799f94ade7a1a527cef793a0154fc6cb3/pyproject.toml

drfho added 3 commits September 3, 2024 22:09

ubuntu-docker: install Zope in editable mode

f315a83

ubuntu-docker: cleaning

79e2107

added log files for docker mouting

9d5bf61

drfho added 3 commits November 21, 2024 07:14

cleaning

32cf652

cleaning

4482aa9

Merge branch 'main' into fb_pip_using_pyproject_toml

59451dd

dwt and others added 2 commits November 26, 2024 11:59

Ensure that the import problem is reproducible

bf3831e

Merge branch 'main' into fb_pip_using_pyproject_toml

a3ba8d6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding pyproject.toml #302

Adding pyproject.toml #302

drfho commented Sep 3, 2024 •

edited

Loading

drfho commented Sep 3, 2024 •

edited

Loading

dwt commented Sep 10, 2024

dwt commented Sep 10, 2024 •

edited

Loading

dwt commented Sep 10, 2024

dwt commented Nov 19, 2024

dwt commented Nov 26, 2024

dwt commented Dec 10, 2024 •

edited by drfho

Loading

Adding pyproject.toml #302

Are you sure you want to change the base?

Adding pyproject.toml #302

Conversation

drfho commented Sep 3, 2024 • edited Loading

drfho commented Sep 3, 2024 • edited Loading

With Zope-editable pyprojects.toml creates resolvable Products paths

dwt commented Sep 10, 2024

dwt commented Sep 10, 2024 • edited Loading

dwt commented Sep 10, 2024

dwt commented Nov 19, 2024

dwt commented Nov 26, 2024

dwt commented Dec 10, 2024 • edited by drfho Loading

drfho commented Sep 3, 2024 •

edited

Loading

drfho commented Sep 3, 2024 •

edited

Loading

dwt commented Sep 10, 2024 •

edited

Loading

dwt commented Dec 10, 2024 •

edited by drfho

Loading