Practical applications and best practices of Metric Learning

[i008]

First of all kudos to the author @KevinMusgrave for this great package.

I don't have any issues or questions directly related to this package it works really well for me so far. What i would want here is to start discussion about real-world problems, practical usages, best practices of the techniques we have in this repo (metric learning etc)
I guess that's the best place to find people involved in such projects?

For instance to start:
I'm working on a cattle face-re-identification problem, im trying to build a robust cow-face-embedding-model that will allow me to do one-shot or few-shot retrieval of cattles without retraining the model (im using a short video to add a cattle to the database, and query single image to retrieve)

I collected a dataset of around 20,000 images with aprox 2,000 unique cattles.

Currently my solutions are build around ideas collected here:
https://github.com/michuanhaohao/reid-strong-baseline
I implemented most of the ideas using p-m-l and lightning

So far I've find it quite hard to get a feel how im really doing. Evaluations of this is hard. For instance i dont have much data of a given ID's from different days, most of the data i have is extracted from videos, which means high correlations between them (frames) , which makes splitting challenging (high risk of over fitting/ leaking)
So i mainly rely on the holdout (ID-s in holdout not present in train) and this works oki'sh but ofcourse has its drawbacks too.

Anyhow some things i'm currently trying to wrap my head around:

• Metric+Classifier vs Just Metric from my observations i get far better results (even on holdout KNN-scores) while training with a classification head then just using triplet learning. My intuition would be that i would at least get close (maybe with much longer training time) but i'm always plateauing at levels much much worse.

• Augmentations, mining and all that. So here i was wondering if i have a heavy augmented mini batch, is it OK to apply different augmentations to the same ID's in the batch? without making this problem too hard for triplet-loss? Anyone has some intuition about this topic in context of metric learning?

• How to handle situations when i have multiple embeddings per ID during inference ( for instance from a video) do people use things like stacking or averaging in practice?

I know many of this can be answered by just experimenting but of-course i have finite resources:)

As i said in the beginning not looking for any particular answers or looking for people to do the hard job for me:D, just looking for people working on similar things to share ideas, experiences etc.

Also if you are just starting a similar project hit me up i'm more then happy to start and share what i already know!

[KevinMusgrave]

Maybe related: https://sites.google.com/view/wacv2020animalreid/home