Can't view/edit/enable sieve filters

[knightsg]

🐛 Bugreport

I enabled sieve in my imap server and also in Cypht with the modules[]=sievefilters config line. I've regenerated my config with config_gen.php but if I log into Cypht and go to the filters page, it's blank as shown below:

I've checked the imap servers settings page and there is no option to enable sieve filters:

In the cypht log there is the following when I access the imap server settings page:

mailserver-cypht-1 | NOTICE: PHP message: Array
mailserver-cypht-1 | (
mailserver-cypht-1 | [0] => Using Hm_PHP_Session with Hm_Auth_IMAP
mailserver-cypht-1 | [1] => Using file based user configuration
mailserver-cypht-1 | [2] => Using sapi: fpm-fcgi
mailserver-cypht-1 | [3] => Request type: HTTP
mailserver-cypht-1 | [4] => Request path: /
mailserver-cypht-1 | [5] => TLS request: 1
mailserver-cypht-1 | [6] => Mobile request: 0
mailserver-cypht-1 | [7] => Page ID: servers
mailserver-cypht-1 | [8] => CACHE backend using: noop
mailserver-cypht-1 | [9] => LOGGED IN
mailserver-cypht-1 | [10] => Handler module Hm_Handler_sieve_filters_enabled activated but not found
mailserver-cypht-1 | [11] => TRANSLATION NOT FOUND :Gandi:
mailserver-cypht-1 | [12] => TRANSLATION NOT FOUND :Kolab Now:
mailserver-cypht-1 | [13] => TRANSLATION NOT FOUND :Office365:
mailserver-cypht-1 | [14] => TRANSLATION NOT FOUND :JMAP Servers:
mailserver-cypht-1 | [15] => TRANSLATION NOT FOUND :Add a JMAP Server:
mailserver-cypht-1 | [16] => TRANSLATION NOT FOUND :Server URL:
mailserver-cypht-1 | [17] => TRANSLATION NOT FOUND :Server URL:
mailserver-cypht-1 | [18] => PHP version 8.2.2
mailserver-cypht-1 | [19] => Zend version 4.2.2
mailserver-cypht-1 | [20] => Peak Memory: 8192
mailserver-cypht-1 | [21] => PID: 31
mailserver-cypht-1 | [22] => Included files: 71
mailserver-cypht-1 | )

My imap server (docker-mailserver with Dovecot) has managesieve enabled and port 4190 exposed to the container running Cypht.

Here is the modules section of my hm3.ini file:

; -----------------------------------------------------------------------------
; Modules
; -----------------------------------------------------------------------------
modules[]=core
; Contacts
; -------
; Contact support. This module requires that at least one "backend" contacts
; module be enabled (ldap_contacts, gmail_contacts, or local_contacts). You
; can enable all the backends you want to support.
modules[]=contacts
modules[]=local_contacts
modules[]=imap
modules[]=sievefilters
; 2 factor authentication
; -----------------------
; This module enables 2 factor authentication using TOTP (compatible with
; Google Authenticator). You must edit the ini file in modules/2fa/ to
; configure a shared secret, then move that file to your app_data_dir.
modules[]=2fa
modules[]=smtp
modules[]=account
modules[]=idle_timer
;modules[]=calendar
modules[]=themes
modules[]=nux
; Simple list of messages read since login
;modules[]=history
; Save and re-run searches easily
;modules[]=saved_searches
; Enable the advanced search form
modules[]=advanced_search
modules[]=highlights
modules[]=profiles
modules[]=inline_message
modules[]=imap_folders
modules[]=keyboard_shortcuts

Version & Environment

Rev: 0760506
OS: Docker v23.0.0 running on Ubuntu 22.04.1 host, Cypht container running docker image php:8.2.2-fpm-alpine3.17

Steps to reproduce

1. Configure modules[]=sievefilters and in hm3 and run scripts/config_gen.php.
2. Log into Cypht.
3. Go to filters page, it will be blank.
4. Also check Servers > Imap Servers page, there is no option to enable sieve filters as shown in screenshot here: Server side filters with Sieve #569.

[marclaporte]

Related for when none of the servers offer Sieve (I am not saying this is your case)
#673

[marclaporte]

If you supply me a test account (send me info in private), I can assign a junior dev to investigate. You can find me in here: https://app.gitter.im/#/room/#cypht-org_community:gitter.im

[josaphatim]

If you supply me a test account (send me info in private), I can assign a junior dev to investigate. You can find me in here: https://app.gitter.im/#/room/#cypht-org_community:gitter.im

I was unaware of this error but I resolved it with this pull request #675

[knightsg]

Thank you for the fix. I can see the "Enable Sieve Filters" option in the Add New IMAP Server settings block. However, I added my IMAP connection via the hm3.ini file, with the imap_auth_name / imap_auth_server / imap_auth_port / imap_auth_tls options, and there is no option to edit that connection to enable the filters. It would be nice to have the ability to configure the enable sieve filters feature through the hm3.ini file as well.

[josaphatim]

However, I added my IMAP connection via the hm3.ini file

Thanks. This is the PR that fixes that: #677

[marclaporte]

@knightsg Please confirm all is good with latest code. Thanks!

[knightsg]

@josaphatim @marclaporte Thank you, the update for the ini file is working. I left a note about the config option naming being wrong in the hm3.sample.ini file in the PR but after changing it in my hm3.ini the sieve config is listed in my imap server config.

However...unfortunately, I now get a totally blank page when I open the filter settings page. There is no PHP error log thrown in my FPM container console, just a generic backend server 500 error: 24/Feb/2023:02:43:20 +0000 "GET /index.php" 500.

I can connect to the sieve port from my Cypht container:

/usr/local/share/cypht # telnet mailserver 4190
Connected to mailserver
"IMPLEMENTATION" "Dovecot (Debian) Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify vnd.dovecot.pipe vnd.dovecot.filter"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN"
"VERSION" "1.0"
OK "Dovecot (Debian) ready."

[marclaporte]

@knightsg FYI: #679

[josaphatim]

@josaphatim @marclaporte Thank you, the update for the ini file is working. I left a note about the config option naming being wrong in the hm3.sample.ini file in the PR but after changing it in my hm3.ini the sieve config is listed in my imap server config.

However...unfortunately, I now get a totally blank page when I open the filter settings page. There is no PHP error log thrown in my FPM container console, just a generic backend server 500 error: 24/Feb/2023:02:43:20 +0000 "GET /index.php" 500.

I can connect to the sieve port from my Cypht container:

/usr/local/share/cypht # telnet mailserver 4190
Connected to mailserver
"IMPLEMENTATION" "Dovecot (Debian) Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify vnd.dovecot.pipe vnd.dovecot.filter"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN"
"VERSION" "1.0"
OK "Dovecot (Debian) ready."

This is the fix for the issue #691

[knightsg]

@josaphatim I've updated to the latest master code and unfortunately I'm still getting the blank filters page. The screenshots below show the servers page with the IMAP server added (from hm3.ini) with the sieve server setting configured and the second shot shows the blank filters page:

https://imgur.com/iU6CBKA
https://imgur.com/N9NEiOW

There's still no useful log output, just the same generic "GET /index.php" 500 error.

[marclaporte]

ok, please join us here: https://app.gitter.im/#/room/#cypht-org_community:gitter.im

We need a dev with server access to debug.

[knightsg]

@marclaporte @josaphatim Apologies, I finally worked out the issue here. I realised php logging wasn't properly enabled in my container (only fpm) which is why I couldn't see the actual server error. I've fixed that and discovered that the issue was that I needed to install PHP sockets module. I did that and it's working now.

Thanks again for your help 👍

[knightsg]

Sorry guys, I have one more issue with this that may or may not be something that can be resolved from within Cypht. As I mentioned in my previous comment, I was able to get this working finally by fixing the issue on my end. However, the stack I was using to test was my local dev environment. When I tried to test the sieve filters on my production setup I found that I still get the blank filters page.

After some investigation I found that it's because I have dovecot configured to use SSL on my production setup. If that's enabled, then when Cypht tries to connect to the managesieve port on the mail server it fails because it uses plain authentication by default. If I disable SSL for my production stack then it works fine.

Do you have any suggestions for how I can resolve this?

Thanks!

[marclaporte]

Interesting. We'll look into it. Is SSL vs non SSL a different port? (like mail servers do for IMAP)

[knightsg]

No, it's the same port as far as I'm aware. I tested a basic connection to the port using telnet and this is the output for both SSL disabled and SSL enabled:

SSL Disabled:

"IMPLEMENTATION" "Dovecot (Debian) Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify vnd.dovecot.pipe vnd.dovecot.filter"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN"
"VERSION" "1.0"

SSL Enabled:

"IMPLEMENTATION" "Dovecot (Debian) Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify vnd.dovecot.pipe vnd.dovecot.filter"
"NOTIFY" "mailto"
"SASL" ""
"STARTTLS"
"VERSION" "1.0"

[marclaporte]

ok, let's see what @josaphatim discovers

[marclaporte]

@knightsg We need @josaphatim to have server access to debug.
#701

[marclaporte]

@knightsg ping :-)

[knightsg]

Sorry, I've been pretty busy lately and forgot to follow up. I'll try and take another look in the next couple of days and see if I still need this open. Jun 15, 2023, 11:31 by ***@***.***:

…

@knightsg <https://github.com/knightsg>> ping :-) — Reply to this email directly, > view it on GitHub <#671 (comment)>> , or > unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFUUWC47SGFOJYHBTT7LI3XLNIIVANCNFSM6AAAAAAU4I7OYA>> . You are receiving this because you were mentioned.> Message ID: > <jasonmunro/cypht/issues/671/1593544933> @> github> .> com>

[marclaporte]

We need you @knightsg :-)

[knightsg]

Ahh, sorry...I totally forgot to follow up on this (again). I have to set up my prod server again as I'm not actually using it yet (just been testing). I'll take a look at that later today and check in with you via the Element chat.

[marclaporte]

@knightsg

[knightsg]

Hi @marclaporte - not sure if you saw but I replied in the chat.

[marclaporte]

Sorry I don't see anything in https://app.gitter.im/#/room/#cypht-org_community:gitter.im

[marclaporte]

For the record, @knightsg @josaphatim and I are solving this in a private chat.

[josaphatim]

ok, let's see what @josaphatim discovers

@marclaporte, @knightsg for now we can pass scheme in the server address something like tls://my.server.com:4190. @kroky fixed here cypht-org/php-sieve-manager@29a9b43

[knightsg]

Hi @josaphatim, I already have that configured in my hm3.ini like below:

imap_auth_sieve_conf_host=tls://mail.mydomain.com:4190

[marclaporte]

Related: cypht-org/cypht-docker#27

[marclaporte]

@knightsg We now use environment variables instead of hm3.ini:
#823

Can you please review to see if the issue persists?

Thanks!

[knightsg]

Hi Marc. I took a look and replaced my existing custom docker image with the prebuilt sailfrog/cypht-docker one, but now there's no sieve option available. I took a look at the list of options on the docker image page and it doesn't list anything there related to sieve. I tried anyway to add a CYPHT_IMAP_AUTH_SIEVE_CONFIG_HOST env var with the same setting I had before but it didn't resolve the issue.

On a side note, I'm note sure how to configure the prebuilt image to use SSL but I guess that's a topic for a new issue ticket.

[marclaporte]

ok, for Docker related issues, we don't have expertise available at the moment. But we have plenty of PHP developers who are moving forward with general Cypht code

[knightsg]

ok, for Docker related issues, we don't have expertise available at the moment. But we have plenty of PHP developers who are moving forward with general Cypht code

That's fair. I'll revert back to my previous setup but update it for the new env variables and see how that goes.

[marclaporte]

The good, the bad and the ugly: We need a Docker expert to guide us!

[marclaporte]

https://github.com/cypht-org/cypht/releases/tag/v2.0.0 has been released

And the next version of Cypht (2.0.1) will include
cypht-org/php-sieve-manager#11
cypht-org/php-sieve-manager#12

[marclaporte]

@knightsg

Cypht 2.x is getting quite good:
https://github.com/cypht-org/cypht/releases/tag/v2.4.0

Cypht 1.4.x doesn't support PHP 8.x
https://github.com/cypht-org/cypht/wiki/Lifecycle

We now have a revamped and official Docker: https://hub.docker.com/r/cypht/cypht

[knightsg]

@knightsg

Cypht 2.x is getting quite good: https://github.com/cypht-org/cypht/releases/tag/v2.4.0

Cypht 1.4.x doesn't support PHP 8.x https://github.com/cypht-org/cypht/wiki/Lifecycle

We now have a revamped and official Docker: https://hub.docker.com/r/cypht/cypht

Thanks for the update! I haven't had much time to spend working on my setup for a while but I've caught some of the comments in the chat and I'm keen to give the latest builds a try.

[knightsg]

@marclaporte Thanks for the heads up, I've come back and had a play with the latest updates and things are looking good! Seems like there's been a lot of work going on so congrats on the team effort 👍

Also, I'm happy to report that after coming back to this and setting everything up from scratch again with the new Cypht docker container I was finally able to get the managesieve connection working. The config I ended up with is as follows in case it's useful:

IMAP_AUTH_NAME=mailserver
IMAP_AUTH_SERVER=mailserver
IMAP_AUTH_PORT=993
IMAP_AUTH_TLS=true
IMAP_AUTH_SIEVE_CONF_HOST=tls://mailserver:4190
IMAP_AUTH_SIEVE_TLS_MODE=true

Initially I had IMAP_AUTH_SIEVE_CONF_HOST=mailserver:4190 (without tls prefixed) and I was getting the error managesieve-login: Login failed: SSL required for authentication: user=<>, method=PLAIN, rip=<redacted>, lip=<redacted>, session=<redacted>. Once I added the tls:// it started working.

I do have one small issue remaining however. When I try to add a filter with the action Move email to mailbox and select one of my IMAP mail folders I get a Server Error popup and in the Cypht log it shows the following:

cypht-1     | NOTICE: PHP message: PHP   6. Hm_Handler_sieve_save_filter->process(['fancy_login_allowed' => TRUE, 'is_mobile' => FALSE, 'changed_settings' => [], 'username' => '<redacted>', 'is_logged' => TRUE, 'data_sources' => [], 'encrypt_ajax_requests' => FALSE, 'encrypt_local_storage' => FALSE, 'default_timezone' => 'UTC', 'enabled_modules' => [0 => 'core', 1 => 'contacts', 2 => 'local_contacts', 3 => 'gmail_contacts', 4 => 'feeds', 5 => 'jmap', 6 => 'imap', 7 => '2fa', 8 => 'smtp', 9 => 'idle_timer', 10 => 'desktop_notifications', 11 => 'calendar', 12 => 'themes', 13 => 'nux', 14 => 'developer', 15 => 'profiles', 16 => 'imap_folders', 17 => 'sievefilters', 18 => 'advanced_search', 19 => 'inline_message', 20 => 'recover_settings', 21 => 'tags'], 'disable_delete_prompt' => FALSE, 'mailto_handler' => TRUE, 'warn_for_unsaved_changes' => TRUE, 'no_password_save' => FALSE, 'language' => 'en', 'date' => '13:00:35', 'http_headers' => ['Content-Language' => 'en', 'X-Frame-Options' => 'SAMEORIGIN', 'X-XSS-Protection' => '1; mode=block', 'X-Content-Type-Options' => 'nosniff', 'Expires' => 'Sun, 19 Nov 2023 21:00:35 GMT', 'Content-Security-Policy' => 'default-src \'none\'; script-src \'self\' \'unsafe-inline\'; connect-src \'self\'; font-src \'self\' https://fonts.gstatic.com; img-src * data:; style-src \'self\' \'unsafe-inline\' https://fonts.googleapis.com;', 'Content-Type' => 'application/json'], 'imap_accounts' => ['673aafd353c54' => [...]], 'site_config' => class Hm_Site_Config_File { protected $source = ''; protected $config = [...]; public $decrypt_failed = FALSE; public $encrypted_str = NULL; public $save_on_login = FALSE; public $user_defaults = [...] }, 'user_config' => class Hm_User_Config_DB { protected $source = ''; protected $config = [...]; public $decrypt_failed = FALSE; public $encrypted_str = NULL; public $save_on_login = FALSE; private $site_config = class Hm_Site_Config_File { ... }; private $dbh = NULL; private $crypt = TRUE; private $username = '<redacted>' }]) /usr/local/share/cypht/lib/modules_exec.php:177
cypht-1     | NOTICE: PHP message: PHP   7. PhpSieveManager\Filters\Actions\BaseFilterAction->__construct($params = ['mailbox' => [0 => 'Junk']]) /usr/local/share/cypht/modules/sievefilters/modules.php:940
cypht-1     | NOTICE: PHP message: PHP   8. PhpSieveManager\Filters\Actions\BaseFilterAction->validateParams() /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php:12
cypht-1     | NOTICE: PHP message: PHP   9. PhpSieveManager\Filters\Actions\BaseFilterAction->validateTypes() /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php:21
cypht-1     | NOTICE: PHP message: PHP Fatal error:  Uncaught PhpSieveManager\Exceptions\FilterActionParamException: Invalid type for parameter: mailbox. Expected Array in /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php:28
cypht-1     | Stack trace:
cypht-1     | #0 /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php(21): PhpSieveManager\Filters\Actions\BaseFilterAction->validateTypes()
cypht-1     | #1 /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php(12): PhpSieveManager\Filters\Actions\BaseFilterAction->validateParams()
cypht-1     | #2 /usr/local/share/cypht/modules/sievefilters/modules.php(940): PhpSieveManager\Filters\Actions\BaseFilterAction->__construct()
cypht-1     | #3 /usr/local/share/cypht/lib/modules_exec.php(177): Hm_Handler_sieve_save_filter->process()
cypht-1     | #4 /usr/local/share/cypht/lib/modules_exec.php(155): Hm_Module_Exec->run_handler_module()
cypht-1     | #5 /usr/local/share/cypht/lib/dispatch.php(218): Hm_Module_Exec->run_handler_modules()
cypht-1     | #6 /usr/local/share/cypht/lib/dispatch.php(188): Hm_Dispatch->process_request()
cypht-1     | #7 /usr/local/share/cypht/site/index.php(53): Hm_Dispatch->__construct()
cypht-1     | #8 {main}
cypht-1     |   thrown in /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php on line 28
cypht-1     | 127.0.0.1 -  19/Nov/2024:21:00:35 +0000 "POST /index.php" 500
cypht-1     | 172.18.0.4 - - [19/Nov/2024:21:00:35 +0000] "POST /?page=sieve_filters HTTP/1.0" 500 1 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:132.0) Gecko/20100101 Firefox/132.0"

The problem appears to be Invalid type for parameter: mailbox. Expected Array in /usr/local/share/cypht/vendor/henrique-borba/php-sieve-manager/src/Filters/Actions/BaseFilterAction.php:28. Is that a legit issue with the code or something I can address at my end?

Anyway, once again thanks for continuing to follow up on my now very old thread 🙂