Authentication error viewing ZMI resource with virtual hosting.

[mauritsvanrees]

This is similar to issue #1195 which has more links, and which has been solved. But the solution from PR #1196 makes a problem when using virtual hosting worse.

Setup:

• Zope 5.9, Plone 6.0.10, Python 3.11.
• Zope runs on port 9341.
• The Plone site is at /plone
• One user account with the Manager role defined on the Zope root, and another defined on the Plone site with Manager role there.
• nginx runs on port 9340 and passes all traffic on to the Plone site object. The main line that does this, is this rewrite rule:

rewrite ^/(.*)$ /VirtualHostBase/http/$host:9340/plone/VirtualHostRoot/$1 last;

With Zope 5.9:

• Zope Manager can access ZMI fine when directly accessing Zope.
• Zope Manager sees ZMI unstyled when going via nginx. I get an Unauthorized on for example http://localhost:9340/++resource++zmi/zmi_base.css
• Plone Manager sees the ZMI unstyled in both cases.

Issue #1195 recognises the last problem and PR #1196 fixes it. I copy the fixed src/App/dtml/manage_page_header.dtml from Zope master and try again:

• Both the Zope and Plone Manager can see the ZMI fine when directly accessing Zope. The Plone user obviously cannot access the Zope root, but on the Plone site the resource urls are correctly like http://localhost:9341/plone/++resource++zmi/zmi_base.css
• Both the Zope and Plone Manager see the ZMI unstyled when going via nginx.

The Plone user has this link in the head:

href="/plone/virtual_hosting///++resource++zmi/zmi_base.css"

The Zope user has this link in the head:

href="/plone/virtual_hosting/++resource++zmi/zmi_base.css"

I have a patch on the copied dtml file that improves this:

< <dtml-let basepath="'/'.join([''] + [p for p in (REQUEST['BASEPATH1'], REQUEST.get('AUTHENTICATION_PATH')) if p])">
---
> <dtml-let authpath="REQUEST.get('AUTHENTICATION_PATH', '').split('virtual_hosting', maxsplit=1)[-1].lstrip('/')"
>           basepath="'/'.join([''] + [p for p in (REQUEST['BASEPATH1'], authpath) if p]).lstrip('/')"
> 	  basepath="'/' + basepath if basepath else ''">

This turns the head links for both users into this when using virtual hosting (and correctly leaves the non-virtual host case unchanged):

href="/++resource++zmi/zmi_base.css"

So this is an improvement for the virtual hosting case.

But the problem still remains: these urls give an Unauthorized when accessed through nginx, for both Manager users:

Unauthorized: http://localhost:9340/++resource++zmi/zmi_base.css
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 181, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 391, in publish_module
  Module ZPublisher.WSGIPublisher, line 269, in publish
  Module ZPublisher.BaseRequest, line 641, in traverse
  Module ZPublisher.HTTPResponse, line 1016, in unauthorized
zExceptions.unauthorized.Unauthorized: You are not authorized to access this resource.

Wait a minute. This is weird. I get that Unauthorized error for four ZMI resources and exactly one minute later, for each of those plus one extra resource, the nginx access log reports this:

*9 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.200.209.1, server: localhost, request: "GET /++resource++zmi/zmi_base.css HTTP/1.1", upstream: "http://127.0.0.1:9346/VirtualHostBase/http/localhost:9340/plone/VirtualHostRoot/++resource++zmi/zmi_base.css", host: "localhost:9340", referrer: "http://localhost:9340/manage_main"

And now Zope hangs. Ah, this is only after I have edited the error_log to not filter out Unauthorized. When I restart and filter it out again, Zope no longer hangs. Maybe this is caused by Products.PDBDebugMode, which is installed in this test site. So never mind that.

Main problem is, for the Zope Manager user:

1. This works: http://localhost:9341/++resource++zmi/zmi_base.css
2. This works: http://localhost:9341/plone/++resource++zmi/zmi_base.css
3. This gives Unauthorized, although it should really be the same as the second one: http://localhost:9340/++resource++zmi/zmi_base.css

Question: why? And how do we fix this?

[mauritsvanrees]

BTW, you get the same authentication errors in a few more DTML files that use BASEPATH1, which were not touched by PR #1196, so they may need adjustment too:

• https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/manage.dtml
• https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/menu.dtml
• https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/copyright.dtml

I wonder if we should define a fixed basepath (or maybe resource_basepath) in one place in Python, so we do the concatenation and fixing of BASEPATH1 and AUTHENTICATION_PATH in one place. I don't know where best to do this though.

[perrinjerome]

I'll try to take a look at this later, unless you or somebody else already wants to fix this. Without looking at the code again, I feel that it might be easier to use some proper request API with understanding of virtual host ( maybe physicalPathToURL ).

Already it looks clear that I fixed only manage_main and forgot about manage (again !)

[perrinjerome]

I have investigated a bit more and I noticed that :

• /manage was not allowed, not because of js and css from ICSSPaths or IJSPaths subscribers, but because it includes some resources from /++resources++zmi/logo/ that are now protected. Now that they are protected ( after d4a2997 ), tjhey are also available from /++resources++logo/ as public ( only the images etc, not the js or css ). For this I suggest changing to include the public resources from /++resources++logo/ here:

Zope/src/App/dtml/manage.dtml

Lines 5 to 11 in c33b799

	<link rel="shortcut icon" type="image/x-icon" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/favicon.ico" />
	<link rel="apple-touch-icon" sizes="180x180" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/apple-touch-icon.png" />
	<link rel="icon" type="image/png" sizes="32x32" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/favicon-32x32.png" />
	<link rel="icon" type="image/png" sizes="16x16" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/favicon-16x16.png" />
	<link rel="manifest" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/site.webmanifest" />
	<link rel="mask-icon" href="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/safari-pinned-tab.svg" color="#5bbad5" />
	<meta name="msapplication-config" content="&dtml-BASEPATH1;/++resource++zmi/logo/favicon/browserconfig.xml"/>

and to only use /++resources++zmi for actually protected resources.

• AUTHENTICATION_PATH is not something to use with virtual hosts, it has a different meaning that "the minimal virtual URL path where a user have permission", it just means something like "the path of the user folder" (and also it looks a bit wrong when virtual host was used). I have experimented with another approach of not adjusting the URLs from the subscribers in dtml, but letting the subscriber produce URLs ready to integrate in HTML, this way we can implement all the logic without the restrictions from DTML. Letting the subscribers produce full URLs seems more flexible, because custom subscribers may provide absolute URLs this way.
• there was another resource here, I changed to serve it using

  Zope/src/App/dtml/menu.dtml

  Line 67 in 1874591

  <script src="&dtml-BASEPATH1;/++resource++zmi/zmi.localstorage.api.js"></script>

Please take a look at #1204 , I have tried all virtual host setups that I could think of in the unit tests, but it would be good if you can confirm it works in your setup.

[mauritsvanrees]

The urls now seem to be generated correctly, thanks.

Initially with your branch I still had the main problem, that http://localhost:9340/++resource++zmi/zmi_base.css gave Unauthorized, with nginx on port 9340 doing a virtual host rewrite/proxy to the Plone site. But after some more restarts it worked for the Zope manager, so that seems to have been some temporary caching problem.
And for the Plone manager it still failed, but that was definitely due to caching: above I simplified the description of the setup a bit, as nginx was actually going to Varnish and Varnish to Plone. When I take Varnish out of the loop, it works for the Plone manager as well.

So I need to have a look at what is wrong in the Varnish setup, but the PR seems fine.

[d-maurer]

Maurits van Rees wrote at 2024-4-4 06:36 -0700:

BTW, you get the same authentication errors in a few more DTML files that use `BASEPATH1`, which were not touched by PR #1196, so they may need adjustment too: * https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/manage.dtml * https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/menu.dtml * https://github.com/zopefoundation/Zope/blob/master/src/App/dtml/copyright.dtml I wonder if we should define a fixed `basepath` (or maybe `resource_basepath`) in one place in Python, so we do the concatenation and fixing of `BASEPATH1` and `AUTHENTICATION_PATH` in one place. I don't know where best to do this though.

I think `ZPublisher.HTTPRequest.HTTPRequest` would be a good place: it already has `getVirtualRoot` and the URL-base should be related.