Best way to train the algorithm?

[derhappy]

We've been using CompreFace in combination with Frigate / Double-Take for some days now and we have many false positives.
For example my wife is mistaken for our 2-year old (89%), our 2- and 5-year olds are mixed up. Even a stranger in a ffp2 mask who initially started being our 5-year old (92%), then turning into our 2-year old (72%) and finally into me (86%).
If I delete and re-train with less images, the results vary but don't get any better.

So my assumption is this that I probably made an error training the detection algorithm.
What is the best way to train it? Should I upload one or multiple images for each person? Should it be a high resolution portrait picture, a biometric flat faced picture or random low quality snapshots? Should I include more realistic pictures where only part of the face is visible, because based on the cameras positions, this is often the case?
Or should I just take some captures of the whole scene with the actual cameras?

Thanks,
Markus

[pospielov]

Hi Markus,

There are two possible ways of facial recognition. The first one is simple, just use CompreFace without additional logic. Then I would recommend:

1. Have high-resolution portrait pictures during training
2. Without additional logic, I would recommend having one picture per person
3. Face should be visible completely

You can improve the accuracy if you add additional logic:

1. You can upload more than 1 example during training. E.g. 5 images per face. When you recognize a face, send prediction_count=5 parameter. It will return 5 most similar examples. So for example you receive such result: face1: 0.99, face2: 0.98, face2: 0.97, face2:0.95, face2: 0.95. The custom logic should understand that probably this is a face2, not face1, and face1 probably is a false positive.
   In this case, it's better to upload faces in different conditions. I would not recommend uploading images with too bad quality though.
2. For even better improvement, if you recognize faces on video, you can add face tracking. I mean there is no big chance of moving face too fast, so if the bounding boxes in different frames are near - probably this is the same person. In this case, you can check the results on the several frames, and decide if this is a false positive. E.g. you have 5 frames and results: face1: 0.97, face1: 0.98, face2: 0.95, face1:0.96, face1: 0.99 - then probably face2 is a false positive.

As a bonus, I found quite an interesting resource:
https://www.areyouyou.eu/
The main idea is to fool face recognition. There are three stages of difficulty.
CompreFace without additional logic is level 1 - you can check yourself that it's quite easy to trick.
But as soon as you add additional logic - it's harder to trick the system.

[derhappy]

Thanks for your quick reply!
I re-trained with only one image now, trying different images per person and always re-evaluating a bunch of pictures. As far as I can tell by now, we all get recognized much less, but there are also much less false positives. I guess there's a trade-off.

I like your ideas about using additional logic. The first one could get problematic if there often are multiple in the frame, but it could definitely help. The second approach sounds even more fail-proof and should probably taken into consideration if there is a tighter integration into a NVR, like Frigate (@blakeblackshear) or maybe a proxy like Double-Take (@jakowenko).

[martinenkoEduard]

Can you share a guide, or a readme on how you retrained compreaface.
I also want to improve accuracy a little bit on my own data.