Using SORT when the number of elements to track is constant and known - ideas to recover missidentification and missmatches

[juanpablo-sanchez]

Hello everybody

I am using SORT for tracking animals placed on closed cages, thus I know the exact number of animals in each cage. I use maskR-CNN for the detection and in many frames it is frequent to missidenfity one, two or even three individuals out of six; also sometimes it happend that althought maskR-CNN detect six, SORT only track 5 or 4 (due to IoU lower than the threshold set). To keep my program running at real time I just can process one frame per second, I do not know whether this low rate of processed frames could be a problem.

I believe I could partially recover the missmatches and the missidentifications. For example if I have 6 detections but just 5 of them match the tracks, the sixth can be asigned by elimination; in the case I would have just 4 matches out of 6 detections, I could solve the unmatches by checking the distance between the centers of the current detections and the previous properly assigned tracks. I mean, I could assing one unmatched (based on IoU) detection to its closest unasigned track. Similarly, if I have less than six detections I could compare the ones I got with those in the tracker and assing to the missing detections the possitions of the tracks in the previous frame that in the current frame did not got any assigned detection, and them update the tracker, considering as new detections some of the previous track positions.

I expect to have properly explained myself about the issue I want to solve. Just want to ask you whether anyone have experience on an issue similar to the one I am proposing here, and whether you can recomend any code allowing me to implement the functionally I have indicated before.

Thank you in advance for your help
Regards
Juan Pablo

[NikoRohr]

Hello,
i am working on an similar project and wonder if you have found a solution in the meantime? I also had the idea to do something like deepsort with an additional classifier after the Re-ID model, but have not tried it yet.

[juanpablo-sanchez]

Hello Niko Still working on it, not much advances, the most significant was to avoid creating new tracks. In a second iteration will review what I have done to see whether I can improve it, them I will share with you. Thank you Juan Pablo Sánchez Serrano Genética y Mejora Animal IRTA Torre Marimón, 08140, Caldes de Montbui, Barcelona Telf.: 934674040 Ext. 1561 Fax : 938650954 ***@***.******@***.***> De: NikoRohr ***@***.***> Enviado el: martes, 17 de mayo de 2022 10:17 Para: abewley/sort ***@***.***> CC: Sanchez, Juan Pablo ***@***.***>; Author ***@***.***> Asunto: Re: [abewley/sort] Using SORT when the number of elements to track is constant and known - ideas to recover missidentification and missmatches (Issue #157) Hello, i am working on an similar project and wonder if you have found a solution in the meantime? I also had the idea to do something like deepsort with an additional classifier after the Re-ID model, but have not tried it yet. — Reply to this email directly, view it on GitHub<#157 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHZCHNTHXCYAWWEIDNXSWFLVKNIYVANCNFSM5SCWE73A>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[ph0ngg]

Hello,
Do you have any solution for your problem, can you share with me
Thank you!

[juanpablo-sanchez]

Hi
I have deteced that the most relevant point is to have a nearly perfect detection. Thus, I have been working on this rather than on struggling with the SORT procedure itself. In the coming weeks will resume with this project, then i will validate whether with an improved detection the trick I was seeking it is relevant or not.
Thank you

[ph0ngg]

thanks for your reply!

[tomfitzrugger]

This is a Re-identification problem. DeepSort features a deep association metric to reduce uncertainty between detections. https://github.com/nwojke/deep_sort. Rapid pose changing can reduce accuracy for DeepSort. ReID using an approach like https://github.com/HanGuangXin/ByteTrack_ReID may be necessary.