Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

versionpurge job getting Job Queue stuck. #1365

Open
cemusta opened this issue May 3, 2022 · 4 comments
Open

versionpurge job getting Job Queue stuck. #1365

cemusta opened this issue May 3, 2022 · 4 comments

Comments

@cemusta
Copy link

cemusta commented May 3, 2022

Gentics Mesh Version, operating system, or hardware.

  • v1.8.2

Operating System

  • Linux, mesh instance dockerized

Problem

  • There are some version purge jobs in the list. When we re-queue these jobs, the cms instance starts a deadlock. This deadlock makes the instance stop responding to API calls.

  • When we restart the instance, that job is still in the queue (not failed). if re-queue another job, the cms instance still takes the first version purge job ( 1st job in the queue ) instead. and the same process happens (deadlock and cms getting not responsive)

  • Because of this, all other jobs got stuck in the queue. We cannot make any schema changes. All schema migration jobs are in the job list and cannot be processed.

problematic job:

		{
			"uuid": "16eadf6a89a8486e9da27546ac854ee7",
			"created": "1970-01-01T00:00:00Z",
			"type": "versionpurge",
			"status": "QUEUED",
			"properties": {
			},
			"completionCount": 0,
			"warnings": [
			]
		}

Reproducer

  • reprocessing jobs via API re-produce the same problem.

Expected behaviour and actual behaviour.

  • Job should either fail or got completed.
@plyhun
Copy link
Contributor

plyhun commented Jul 5, 2022

Hi @cemusta ,
Thanks for the ticket. Do you have any chance to get the Mesh logs? Normally on Mesh restart all the queued jobs start, and if something happen, this is usually depicted in the logs.
We have also fixed one issue regarding the job stability, releasing it as https://github.com/gentics/mesh/releases/tag/1.8.5 / - couldn't you please check if this version solves you problem?
Thanks in advance!

@cemusta
Copy link
Author

cemusta commented Jul 5, 2022

Hi,

We will update to 1.8.5 and update this issue afterward.

Best,
Cem

@plyhun
Copy link
Contributor

plyhun commented Jul 11, 2022

Mesh v1.8.6 is recently released, targeting more job improvements - couldn't you please try it?

@cemusta
Copy link
Author

cemusta commented Jul 12, 2022

Will try and update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants