Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): refresh face detection #12335

Merged
merged 4 commits into from
Oct 4, 2024
Merged

Conversation

mertalev
Copy link
Contributor

@mertalev mertalev commented Sep 5, 2024

Description

This PR allows re-running face detection without clearing face data. This can be done on one or more assets with the "Refresh faces" option, or for all assets in the job panel with a new Refresh button.

It compares existing and new bounding boxes for an asset with the IoU metric. If a new bounding box is above the IoU threshold, it's considered a duplicate of an existing face and skipped. If an existing face doesn't appear in the new set of bounding boxes, it's removed.

This IoU-based matching has an additional benefit in allowing faces from non-machine learning sources to be associated with an embedding. This means that detected faces can match against people listed in face metadata instead of creating duplicate people.

To make the three options of Reset / Refresh / Missing work with the current API, a distinction is made between force being undefined and false. The former maps to Refresh, and the latter maps to Missing.

face-detection-job-panel

refresh-faces

Testing:

  • Increasing the detection threshold and re-running removes existing faces below the new threshold
  • Lowering the threshold and re-running adds new faces
  • Refreshing with the same face detection settings doesn't change anything
  • "Missing" and "All" (now called "Reset") behave the same
  • Refreshing faces for a single asset works as expected

@mertalev mertalev changed the title feat(server,web): refresh face detection feat(server): refresh face detection Sep 7, 2024
@mertalev mertalev force-pushed the feat/rerun-face-detection branch 2 times, most recently from 84caf4d to b23d5f5 Compare September 7, 2024 14:15
@pyorot
Copy link
Contributor

pyorot commented Sep 7, 2024

seems like time to put all the Refresh… options in a sub-menu

@radh21301
Copy link

radh21301 commented Sep 8, 2024

Hey. Would this also work when we change the facial recognition model? Like from buffalo_l to antelopev2. I suppose not?

@mertalev
Copy link
Contributor Author

mertalev commented Sep 8, 2024

No, there are a few more things that would need to happen to let you switch without resetting everything. But it's a big step toward that.

@sammyke007
Copy link

Will this be in the next update?

handle non-ml faces
@mertalev mertalev marked this pull request as ready for review October 3, 2024 23:34
Copy link
Contributor

@jrasm91 jrasm91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Can you add a TODO to not use undefined in the API (for force)? Imo we should migrate to a more explicit API in the future instead of using undefined for a third state.

@mertalev
Copy link
Contributor Author

mertalev commented Oct 4, 2024

Do you know how to make the PR label validation check happy?

@mertalev mertalev merged commit 2c87683 into main Oct 4, 2024
35 of 36 checks passed
@mertalev mertalev deleted the feat/rerun-face-detection branch October 4, 2024 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants