Missing Jobs in Large Batch Rendering

[fatihmert]

Describe the bug
When processing a large batch of jobs (e.g., 300), nexrender occasionally skips some jobs (e.g., 20 out of 300). Smaller batches (5–10 jobs) work fine without any issues. This skipping may be related to the ffmpeg process not completing before the next job starts, but I am not entirely sure.

I have temporarily resolved the issue by introducing a delay (await sleep(2500)) between jobs, which ensures they are processed sequentially. However, this is a workaround, and I believe the issue could be addressed at the core level.

Steps to Reproduce

1. Create a batch of jobs (e.g., 300) using the provided sample code and JSON configuration.
2. Process the jobs without adding any delay between them.
3. Observe that some jobs are skipped.

Example Code

for (let i = 0; i < LIMIT_PROJECTS; i++) {
    const domainName = Object.keys(AEProjectPathes)[i];
    const automationPathGroup = Object.values(AEProjectPathes)[i];

    for (let j = 0; j < LIMIT_VIDEOS; j++) {
        const videoItem = videos[j];

        const payload = {
            AE_PROJECT_PATH: projectFilePath(automationPathGroup[0]),
            INPUT_VIDEO_PATH: videoItem.input,
            OUTPUT_VIDEO_PATH: renderName(videoItem, domainName),
            TITLE: videoItem.title,
            FONT_SIZE: videoItem.fontSize,
            BASELINE_SHIFT: videoItem.baselineShift,
            MASTER_INDEX_ON_BUNDLE: automationPathGroup[1]
        };

        const videoJob = compileJobTemplate(payload);

        try {
            await render(videoJob, { debug: DEBUG });
        } catch (error) {
            console.error(error);
        }

        await sleep(2500); // workaround to prevent skipping jobs
    }
}

Example Job JSON:

{
  "template": {
    "src": "file://automation.aep",
    "composition": "Bundle",
    "outputModule": "H.264 - Match Render Settings - 15 Mbps",
    "outputExt": "mp4",
    "settingsTemplate": "Best Settings"
  },
  "assets": [
    {
      "type": "data",
      "composition": "MASTER->TITLE",
      "layerIndex": 5,
      "property": "Source Text",
      "value": "Sample Text"
    },
    {
      "type": "data",
      "composition": "MASTER->TITLE",
      "layerIndex": 5,
      "property": "Source Text.fontSize",
      "value": 88
    },
    {
      "type": "data",
      "composition": "MASTER->TITLE",
      "layerIndex": 5,
      "property": "Source Text.baselineShift",
      "value": 40
    },
    {
      "type": "video",
      "composition": "MASTER",
      "layerIndex": 5,
      "src": "file://test.mp4",
      "name": "presenter",
      "useOriginal": true
    },
    {
      "type": "script",
      "src": "file://adjust-duration.jsx",
      "parameters": [
        {
          "key": "videoPath",
          "value": "test.mp4"
        },
        {
          "key": "durationOffset",
          "value": 10.0
        },
        {
          "key": "masterIndexOnBundle",
          "value": 5
        }
      ]
    }
  ],
  "actions": {
    "postrender": [
      {
        "module": "@nexrender/action-encode",
        "preset": "mp4",
        "output": "output.mp4",
        "params": {
          "-acodec": "aac",
          "-ab": "128k",
          "-ar": "44100",
          "-vcodec": "libx264",
          "-r": "25"
        }
      }
    ]
  }
}

Information about environment

• nexrender version: 1.57.4
• Setup: CLI
• Installation method: Globally installed npm package
• Operating System: Apple M2 Sonoma 14.4.1
• Additional Information: The issue seems to occur only with larger batch sizes (e.g., 300 jobs).

Expected behavior
All jobs in the batch should process sequentially without skipping any.

Screenshots
Not applicable, as the issue is not visually apparent.

[inlife]

Hey there,
Could you provide any logs related to the skips? For all my time working with Nexrender, both locally and in the cloud, I have not seen that happen. It would be interesting to know more.