How to do the relation inference with custom video?

[Puqi7]

Hi, I have followed your steps in stage 1 (IPS) and successfully got the segmentation and unitrack results.

However, when I follow the instructions on the preparation for the relation model (# Prepare for relation modeling: sh scripts/utils/prepare_rel_set.sh). I found that it requires the ground truth of relation to generate relation.pickle . If without relation.pickle, the following steps for relation inference (# Test sh scripts/test/test_relation_full.sh) could not work.

I wanner to know how could I get the relation prediction on my custom dataset without the gt relation (only: video.mp4+segmation results(frames mask)+unitrack results(qurey_feats.pickle)).

[Jingkang50]

In test_relation_full.sh, relation.pickle seems not necessary for relation inference, but only for evaluation. You may remove the code around relation.pickle in test_relation_full.py?

[Puqi7]

Thanks for your quick response! I used query_feat.pickle as ['feats'] input instead of relation.pickle
I have another questions: Why the relation results.txt of the relation model(# Test sh scripts/test/test_relation_full.sh) only contains the subject, relation, object but no time information. (for example: from frame x- frame y)

[Puqi7]

Thanks for your amazing code! I have found the frame_id in relation_span.