Face detection

[whyboris]

It seems like it may be relatively simple to perform face detection (placing a bounding box across any face found in an image). This process may be super fast: apparently faster than realtime for video (which means stupid-fast for a few screenshots).

It may be simple enough to simply detect any faces within a filmstrip and extract them into a new faces array which can be viewed separately somehow (would need UI work -- unsure what it would look like yet).

Possibly the best solution: https://github.com/tehnokv/picojs

Another alternative is: https://github.com/eduardolundgren/tracking.js but it hasn't been updated since 2018 😕

Another alternative is: https://github.com/justadudewhohacks/face-api.js/ (see Tiny Face Detector), but it requires TensorFlow and I don't want some heavy dependencies for this minor feature 😅

Someday in far future, we could get very advanced with face recognition too: https://github.com/justadudewhohacks/face-api.js/

🤔 🤷‍♂ 😁

[whyboris]

Update:

• Tried picojs and it's not a good fit: it has many false-positives (e.g. finds faces in elbows)
• Tried face-api and I'm really liking it. Assuming including TensorFlow and other sub-dependencies isn't going to make the app too heavy, I just might include it. We'll see 🤷‍♂

I have a script that seems to generate an image strip with faces found in all the extracted screenshots. I'm thinking create a new view that looks like the filmstrip view, but it will only show faces 🤔

I'm open for ideas about how to display the found faces and what to do with them. 👂

[cal2195]

Well this sounds very exciting! 😄

Two thoughts about what you could do with the data:

• If you can find the same face in multiple videos, this would be a very cool improvement to the similar video recommendations (Right click -> Show more videos with this actor)

• Secondly, I see you can also detect expressions - this would be a very cool tag to add, if the user enabled it. (eg. tag videos with happy, surprised etc.)

I like this direction a lot! 🎉

[whyboris]

I have a repository for the internal functionality: https://github.com/whyboris/extract-faces-node

You can take any thumbnail strip from your library and run extract faces.

One caveat is you have to set the number of screenshots there are in the filmstrip inside the script:
https://github.com/whyboris/extract-faces-node/blob/master/src/pipeline.ts#L8

🚀 I'll get a branch of VHA ready to see how well it plays with Electron 😅

[whyboris]

Currently running into trouble when trying to install canvas 😓 -- so far no-go ...
Electron post-install script errors out not being able to find a pre-built binary(?)

[whyboris]

Problem solved 🎉 😂 whyboris/extract-faces-node#1 -- was able to remove canvas 😁
Attempting again with VHA 🤞

[whyboris]

Seems to work well 😎 #342
App compiles to 125MB 👌
Finds and extracts faces while in dev but not yet from the built app -- I'll investigate 🔍 😁 🔎

[frez777]

Pretty cool idea!

Well, even the idea of adding an actor section itself is very useful and nice feature.

[whyboris]

@frez777 -- btw -- have you clicked on the "Auto Tags"? It will automatically find all the 2-word strings that appear in your files (generally actor names if they appear in the file names).

I'll likely expand this feature more -- thank you for the request 👍