Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix delete_old_otks job on worker deployments #17960

Open
wants to merge 1 commit into
base: release-v1.120
Choose a base branch
from

Conversation

richvdh
Copy link
Member

@richvdh richvdh commented Nov 25, 2024

In a worker-mode deployment, the E2eKeysHandler is not necessarily loaded, which means the handler for the delete_old_otks task will not be registered. Make sure we load the handler.

Introduced in #17934

In a worker-mode deployment, the `E2eKeysHandler` is not necessarily loaded,
which means the handler for the `delete_old_otks` task will not be
registered. Make sure we load the handler.
@richvdh richvdh requested a review from a team as a code owner November 25, 2024 16:07
@richvdh
Copy link
Member Author

richvdh commented Nov 25, 2024

Without this, it fails repeatedly with:

2024-11-25 16:08:01,943 - synapse.metrics.background_process_metrics - 253 - ERROR - launch_scheduled_tasks-97870 - Background process 'launch_scheduled_tasks' threw an exception
Capture point (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/synapse/src/synapse/app/generic_worker.py", line 367, in <module>
    main()
  File "/home/synapse/src/synapse/app/generic_worker.py", line 363, in main
    start(sys.argv[1:])
  File "/home/synapse/src/synapse/app/generic_worker.py", line 358, in start
    _base.start_worker_reactor("synapse-generic-worker", config)
  File "/home/synapse/src/synapse/app/_base.py", line 139, in start_worker_reactor
    start_reactor(
  File "/home/synapse/src/synapse/app/_base.py", line 204, in start_reactor
    run()
  File "/home/synapse/src/synapse/app/_base.py", line 186, in run
    run_command()
  File "/home/synapse/src/synapse/app/_base.py", line 124, in <lambda>
    run_command: Callable[[], None] = lambda: reactor.run(),
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/base.py", line 701, in run
    self.mainLoop()
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/base.py", line 709, in mainLoop
    self.runUntilCurrent()
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/base.py", line 1063, in runUntilCurrent
    f(*a, **kw)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 873, in callback
    self._startRunCallbacks(result)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 980, in _startRunCallbacks
    self._runCallbacks()
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 1074, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 1960, in _gotResultInlineCallbacks
    _inlineCallbacks(r, gen, status, context)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 2112, in _inlineCallbacks
    status.deferred.callback(callbackValue)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 873, in callback
    self._startRunCallbacks(result)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 980, in _startRunCallbacks
    self._runCallbacks()
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 1074, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 873, in callback
    self._startRunCallbacks(result)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 980, in _startRunCallbacks
    self._runCallbacks()
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 1074, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 1960, in _gotResultInlineCallbacks
    _inlineCallbacks(r, gen, status, context)
  File "/home/synapse/env-python311/lib/python3.11/site-packages/twisted/internet/defer.py", line 2014, in _inlineCallbacks
    result = context.run(gen.send, result)
Traceback (most recent call last):
  File "/home/synapse/src/synapse/metrics/background_process_metrics.py", line 251, in run
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/synapse/src/synapse/util/task_scheduler.py", line 339, in _launch_scheduled_tasks
    await self._launch_task(task)
  File "/home/synapse/src/synapse/util/task_scheduler.py", line 392, in _launch_task
    raise Exception(
Exception: No function associated with action delete_old_otks of the scheduled task delete_old_otks_task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant