Workflow for video images?

[snurrebassen]

I just managed to get Clipseg to work for still images, and it’s such a neat tool! I’m trying to set up a workflow to generate masks for video clips too.

First of all, I’m completely new to this so apologize if this is a plain silly idea for this particular custom node, but how would one go about to load a video instead of just a single image through the clipseg node?

When trying this, I get an error saying the "input array is receiving more dimensions than expected”:

Error occurred when executing CLIPSeg:

Too many dimensions: 4 > 3.

File "E:\COMFYUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\custom_nodes\clipseg.py", line 118, in segment_image
i = Image.fromarray(image_np, mode="RGB")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\snurrebassen\AppData\Local\Programs\Python\Python312\Lib\site-packages\PIL\Image.py", line 3103, in fromarray
raise ValueError(msg)

I was wondering if anyone has suggestions or workarounds for this issue.

[yegods]

did you ever figure out your problem with this? I'm having the same issue, working with video.

[snurrebassen]

did you ever figure out your problem with this? I'm having the same issue, working with video.

Oof been a while since i was working with these things, but iirc I found a workaround inputting frame by frame separately instead of a video stream, just running it as a batch

[yegods]

thanks. i have come to a similar conclusion... the clipseg input seems to expect a single image, and it is receiving an image batch or something from the load video node. funny that the other nodes like ksampler and such don't seem to have an issue with this.