Out of memory issue - Media analysis job

[mahalakshme]

Issue:

etl jobs were still running after this happened.. seems to only have affected media analysis job and the main service which has the /ping endpoint

Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: 2024-10-16T08:55:38.415Z ERROR 92062 --- [ni ETL_Worker-1] org.quartz.core.ErrorLogger              : Job (MediaAnalysisJobs.7e8f3e7f-1c9e-4a35-b6f4-fdd4eb30bfb3>
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: java.lang.OutOfMemoryError: Java heap space

Stack Trace

Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: Caused by: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         ... 1 more
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: Caused by: org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Java heap space]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.quartz.core.JobRunShell.run(JobRunShell.java:218)
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: org.quartz.JobExecutionException: org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Java heap s>
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: 2024-10-16 08:55:38,415 ERROR Avni ETL_Worker-1 [etl.service.EtlService] Error in job run
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: Caused by: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) ~[quartz-2.3.2.jar!/:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.quartz.core.JobRunShell.run(JobRunShell.java:213) ~[quartz-2.3.2.jar!/:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: org.quartz.SchedulerException: Job threw an unhandled exception.
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: 2024-10-16T08:55:38.415Z ERROR 92062 --- [ni ETL_Worker-1] org.quartz.core.ErrorLogger              : Job (MediaAnalysisJobs.7e8f3e7f-1c9e-4a35-b6f4-fdd4eb30bfb3>
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: 2024-10-16T08:55:38.415Z ERROR 92062 --- [ni ETL_Worker-1] org.quartz.core.JobRunShell              : Job MediaAnalysisJobs.7e8f3e7f-1c9e-4a35-b6f4-fdd4eb30bfb3 >
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: Caused by: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-10.1.8.jar!/:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.apache.catalina.core.StandardServer.lambda$startInternal$0(StandardServer.java:918) ~[tomcat-embed-core-10.1.8.jar!/:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at org.apache.catalina.core.StandardServer.startPeriodicLifecycleEvent(StandardServer.java:930) ~[tomcat-embed-core-10.1.8.jar!/:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]:         at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:na]
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: 2024-10-16T08:55:38.394Z ERROR 92062 --- [alina-utility-3] o.apache.catalina.core.StandardServer    : Error sending periodic event
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: java.lang.OutOfMemoryError: Java heap space
Oct 16 08:55:38 ip-10-100-1-136 start.sh[92062]: Exception in thread "http-nio-8022-Poller" Exception in thread "Catalina-utility-1" Exception in thread "HikariPool-1 housekeeper" java.lang.OutOfMemoryError: Ja>
Oct 16 08:55:28 ip-10-100-1-136 start.sh[92062]: Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Catalina-utility-2"