404 error via playbook-managed-traefik

[whowantsmybigdata]

Describe the bug
This is the only error-logs I can find when running the playbook since last successful update on 10th of November. I just tried the next git pull on the 18th and since then I had this issue. Rolling back solves the problem completely without changing anything else.

 containerd[716]: time="2024-11-20T01:38:22.631593980+01:00" level=warning msg="error from *cgroupsv2.Manager.EventChan" error="failed to add inotify watch for \"/sys/fs/cgroup/system.slice/docker-1f71c4018361063c39e9b7d104e87ff9b3d6a69c54a92d5edfe4f3874d8c2dce.scope/memory.events\": no such file or directory"
matrix-postmoogle[1663145]: INF app/cmd/postmoogle/main.go:48 > #############################
 matrix-postmoogle[1663145]: INF app/cmd/postmoogle/main.go:49 > Postmoogle
 matrix-postmoogle[1663145]: INF app/cmd/postmoogle/main.go:50 > Matrix: true
 matrix-postmoogle[1663145]: INF app/cmd/postmoogle/main.go:51 > #############################
 matrix-postmoogle[1663145]: INF app/vendor/go.mau.fi/util/dbutil/log.go:78 > Database is up to date component=crypto current_version=7 db_section=matrix_state latest_known_version=7 oldest_compatible_version=3
 matrix-postmoogle[1663145]: INF app/vendor/go.mau.fi/util/dbutil/log.go:78 > Database is up to date component=crypto current_version=15 db_section=crypto latest_known_version=15 oldest_compatible_version=15
 matrix-synapse[1642757]: 2024-11-20 00:38:22,759 - shared_secret_authenticator - 102 - INFO - POST-189 - Authenticating user `postmoogle` with login type `m.login.password`
 matrix-synapse[1642757]: 2024-11-20 00:38:22,760 - shared_secret_authenticator - 113 - INFO - POST-189 - Bad hmac value for user: @postmoogle:{domain-hidden}}

and

matrix-traefik[1641370]: 213.47.188.245 - - [20/Nov/2024:00:41:22 +0000] "POST /_matrix/client/v3/keys/query HTTP/2.0" 404 19 "-" "-" 303 "-" "-" 0ms
Nov 20 01:41:22 Pi4 matrix-postmoogle[1667522]: FTL app/cmd/postmoogle/main.go:126 > cannot initialize matrix bot error="failed to query own keys to make sure device is properly configured: failed to POST /_matrix/client/v3/keys/query: HTTP 404"

To Reproduce

My `vars.yml` file looks like this:

matrix_architecture: arm64

matrix_domain: {domain-hidden}
matrix_static_files_container_labels_base_domain_enabled: true

matrix_coturn_turn_static_auth_secret: {hidden}
matrix_synapse_turn_uris:
- turns:matrix.{domain-hidden}?transport=udp
- turns:matrix.{domain-hidden}?transport=tcp
- turn:matrix.{domain-hidden}?transport=tcp
- turn:matrix.{domain-hidden}?transport=udp

matrix_coturn_denied_peer_ips:
  - 0.0.0.0-0.255.255.255
#  - 10.0.0.0-10.255.255.255
  - 100.64.0.0-100.127.255.255
  - 127.0.0.0-127.255.255.255
  - 169.254.0.0-169.254.255.255
#  - 172.16.0.0-172.31.255.255
  - 192.0.0.0-192.0.0.255
  - 192.0.2.0-192.0.2.255
  - 192.88.99.0-192.88.99.255
#  - 192.168.0.0-192.168.255.255
  - 198.18.0.0-198.19.255.255
  - 198.51.100.0-198.51.100.255
  - 203.0.113.0-203.0.113.255
  - 240.0.0.0-255.255.255.255
  - ::1
  - 64:ff9b::-64:ff9b::ffff:ffff
  - ::ffff:0.0.0.0-::ffff:255.255.255.255
  - 100::-100::ffff:ffff:ffff:ffff
  - 2001::-2001:1ff:ffff:ffff:ffff:ffff:ffff:ffff
  - 2002::-2002:ffff:ffff:ffff:ffff:ffff:ffff:ffff
  - fc00::-fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
  - fe80::-febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff


matrix_synapse_macaroon_secret_key: {hidden}
matrix_homeserver_generic_secret_key: "{{ matrix_synapse_macaroon_secret_key }}"
matrix_synapse_ext_password_provider_shared_secret_auth_enabled: true
matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret: {hidden}
matrix_synapse_max_upload_size_mb: 10000
matrix_synapse_url_preview_enabled: false
matrix_appservice_double_puppet_enabled: true

matrix_playbook_reverse_proxy_type: playbook-managed-traefik
traefik_config_certificatesResolvers_acme_enabled: false
traefik_ssl_dir_enabled: true
traefik_configuration_extension_yaml: |
  providers:
    file:
     filename: /config/certificates.yml
     watch: true
aux_file_definitions:
  - dest: "{{ traefik_ssl_dir_path }}/privkey.pem"
    src: "/etc/letsencrypt/live/{domain-hidden}/privkey.pem"
        # Alternatively, comment out `src` above and uncomment the lines below to provide the certificate content inline.
#    content: |

  - dest: "{{ traefik_ssl_dir_path }}/cert.pem"
    src: "/etc/letsencrypt/live/{domain-hidden}/fullchain.pem"
        # Alternatively, comment out `src` above and uncomment the lines below to provide the certificate content inline.
#    content: |

  # The `/ssl/..` paths below are in-container paths, not paths on the host (/`matrix/traefik/ssl/..`). Do not change them!
  - dest: "{{ traefik_config_dir_path }}/certificates.yml"
    content: |
      tls:
        certificates:
          - certFile: /ssl/cert.pem
            keyFile: /ssl/privkey.pem
        stores:
          default:
            defaultCertificate:
              certFile: /ssl/cert.pem
              keyFile: /ssl/privkey.pem

postgres_connection_password: {hidden}

matrix_registration_enabled: true
matrix_registration_admin_secret: {hidden}
matrix_registration_configuration_extension_yaml: |
   password:
     min_length: 12

matrix_synapse_admin_enabled: true

#exim_relay_enabled: false
#exim_relay_sender_address: ""
exim_relay_relay_use: true
exim_relay_relay_host_name: "smtp-relay.brevo.com"
exim_relay_relay_host_port: 587
exim_relay_relay_auth: true
exim_relay_relay_auth_username: "{hidden}"
exim_relay_relay_auth_password: "{hidden}"


matrix_mautrix_signal_enabled: true
matrix_mautrix_signal_relaybot_enabled: true
matrix_mautrix_signal_configuration_extension_yaml: |
  #network:
    #use_contact_avatars: true
  bridge:
    permissions:
      '@{username-hidden}:{domain-hidden}': admin
    relay:
      admin_only: false
      enabled: true
    #autocreate_contact_portal: true
    #private_chat_portal_meta: true
  encryption:
    allow: true
    default: true
    appservice: false

matrix_postmoogle_enabled: true
matrix_postmoogle_password: {hidden}
matrix_postmoogle_ssl_path: "/etc/letsencrypt"
matrix_postmoogle_tls_cert: "ssl/live/{domain-hidden}/fullchain.pem"
matrix_postmoogle_tls_key: "ssl/live/{domain-hidden}/privkey.pem"
matrix_postmoogle_tls_required: true
matrix_postmoogle_admins:
  - "@{username-hidden}:{{ matrix_domain }}"

matrix_mautrix_discord_enabled: true
matrix_mautrix_discord_configuration_extension_yaml: |
  bridge:
    permissions:
      '@{username-hidden}:{domain-hidden}': admin
    encryption:
      allow: true
      default: true

Matrix Server:

• OS: Raspbian 12
• Architecture: arm64

Ansible:
If your problem appears to be with Ansible, tell us:

• where you run Ansible: on the Matrix-server itself
• what version of Ansible: ansible [core 2.14.16] python version = 3.11.2 jinja version = 3.1.2

Additional context
I know there is another closed issue with compression-middleware and self-managed ssl-certs but I set the certs as mentioned in the docs and the issue and my error doesnt mention compress-middleware.

[Keeberos]

That's where your problem lies. #3778
I didn't get it at first, but then I did... You have old certificate description variables. specifically this one - traefik_configuration_extension_yaml should be changed to traefik_provider_configuration_extension_yaml according to the instructions on how to use your certificates. https://github.com/spantaleev/matrix-docker-ansible-deploy/docs/configuring-playbook-ssl-certificates.md
#3778

[whowantsmybigdata]

oh thanks for the reply. missed the change to traefik_provider_configuration_extension_yaml variable.
Anyway now when setting everything as mentioned in the docs, I get the playbook to run without getting 404 errors but instead I have invalid certificates.
When looking inside the matrix-traefik-container for the /ssl/cert.pem and /ssl/privkey.pem and the /config/provider.yml /config/certifcates.yml and /config/traefik.yml everything seemed to have worked (the certs are uploaded via aux and the config points to the correct direction) but when trying to open e.g. element-web its marked as unsecure because of "TRAEFIK DEFAULT CERT" only...

[spantaleev]

Only a single file provider's configuration is loaded (most likely provider.yml).

You shouldn't need your certificates.yml file. Hopefully, you've migrated its settings to traefik_provider_configuration_extension_yaml.

---

Still.. you may be hitting some other issue if you're seeing the default certificate.

[whowantsmybigdata]

You shouldn't need your certificates.yml file. Hopefully, you've migrated its settings to traefik_provider_configuration_extension_yaml.

Maybe I misunderstood but I didn't use any extra settings besides the certs. I just copy and pasted what is given in the docs and changed the src paths to the paths I had before (as seen in my vars.yml above). Unfortunately traefik seems to ignore the certs that way.
Maybe I should try to completely remove the docker image so it sets everything up from scratch later...