Detect rotated faces

[fengweiqiang]

The number of faces in this image should be 4 rectangles, so why get -out file is 3 rectangles (-iou 0.1 or -iou 0.2)

-iou 0.1

-iou 0.2

[esimov]

This issue fits into the todo list: right now the API cannot detect the rotated faces. Please check the documentation. In this particular case the girl face exceeds the maximum permissible threshold which the algorithm can detect faces.

[fengweiqiang]

I used https://github.com/hybridgroup/gocv
Img := gocv.IMRead("./facelearn/duogeren.jpg", gocv.IMReadReducedColor2)
He called the C library and can get 4 rectangles. I don't know if it can help you.

type IMReadFlag int

const (
	// IMReadUnchanged return the loaded image as is (with alpha channel,
	// otherwise it gets cropped).
	IMReadUnchanged IMReadFlag = -1

	// IMReadGrayScale always convert image to the single channel
	// grayscale image.
	IMReadGrayScale = 0

	// IMReadColor always converts image to the 3 channel BGR color image.
	IMReadColor = 1

	// IMReadAnyDepth returns 16-bit/32-bit image when the input has the corresponding
	// depth, otherwise convert it to 8-bit.
	IMReadAnyDepth = 2

	// IMReadAnyColor the image is read in any possible color format.
	IMReadAnyColor = 4

	// IMReadLoadGDAL uses the gdal driver for loading the image.
	IMReadLoadGDAL = 8

	// IMReadReducedGrayscale2 always converts image to the single channel grayscale image
	// and the image size reduced 1/2.
	IMReadReducedGrayscale2 = 16

	// IMReadReducedColor2 always converts image to the 3 channel BGR color image and the
	// image size reduced 1/2.
	IMReadReducedColor2 = 17

	// IMReadReducedGrayscale4 always converts image to the single channel grayscale image and
	// the image size reduced 1/4.
	IMReadReducedGrayscale4 = 32

	// IMReadReducedColor4 always converts image to the 3 channel BGR color image and
	// the image size reduced 1/4.
	IMReadReducedColor4 = 33

	// IMReadReducedGrayscale8 always convert image to the single channel grayscale image and
	// the image size reduced 1/8.
	IMReadReducedGrayscale8 = 64

	// IMReadReducedColor8 always convert image to the 3 channel BGR color image and the
	// image size reduced 1/8.
	IMReadReducedColor8 = 65

	// IMReadIgnoreOrientation do not rotate the image according to EXIF's orientation flag.
	IMReadIgnoreOrientation = 128

	//IMWriteJpegQuality is the quality from 0 to 100 for JPEG (the higher is the better). Default value is 95.
	IMWriteJpegQuality = 1

	// IMWriteJpegProgressive enables JPEG progressive feature, 0 or 1, default is False.
	IMWriteJpegProgressive = 2

	// IMWriteJpegOptimize enables JPEG optimization, 0 or 1, default is False.
	IMWriteJpegOptimize = 3

	// IMWriteJpegRstInterval is the JPEG restart interval, 0 - 65535, default is 0 - no restart.
	IMWriteJpegRstInterval = 4

	// IMWriteJpegLumaQuality separates luma quality level, 0 - 100, default is 0 - don't use.
	IMWriteJpegLumaQuality = 5

	// IMWriteJpegChromaQuality separates chroma quality level, 0 - 100, default is 0 - don't use.
	IMWriteJpegChromaQuality = 6

	// IMWritePngCompression is the compression level from 0 to 9 for PNG. A
	// higher value means a smaller size and longer compression time.
	// If specified, strategy is changed to IMWRITE_PNG_STRATEGY_DEFAULT (Z_DEFAULT_STRATEGY).
	// Default value is 1 (best speed setting).
	IMWritePngCompression = 16

	// IMWritePngStrategy is one of cv::IMWritePNGFlags, default is IMWRITE_PNG_STRATEGY_RLE.
	IMWritePngStrategy = 17

	// IMWritePngBilevel is the binary level PNG, 0 or 1, default is 0.
	IMWritePngBilevel = 18

	// IMWritePxmBinary for PPM, PGM, or PBM can be a binary format flag, 0 or 1. Default value is 1.
	IMWritePxmBinary = 32

	// IMWriteWebpQuality is the quality from 1 to 100 for WEBP (the higher is
	// the better). By default (without any parameter) and for quality above
	// 100 the lossless compression is used.
	IMWriteWebpQuality = 64

	// IMWritePamTupletype sets the TUPLETYPE field to the corresponding string
	// value that is defined for the format.
	IMWritePamTupletype = 128

	// IMWritePngStrategyDefault is the value to use for normal data.
	IMWritePngStrategyDefault = 0

	// IMWritePngStrategyFiltered is the value to use for data produced by a
	// filter (or predictor). Filtered data consists mostly of small values
	// with a somewhat random distribution. In this case, the compression
	// algorithm is tuned to compress them better.
	IMWritePngStrategyFiltered = 1

	// IMWritePngStrategyHuffmanOnly forces Huffman encoding only (no string match).
	IMWritePngStrategyHuffmanOnly = 2

	// IMWritePngStrategyRle is the value to use to limit match distances to
	// one (run-length encoding).
	IMWritePngStrategyRle = 3

	// IMWritePngStrategyFixed is the value to prevent the use of dynamic
	// Huffman codes, allowing for a simpler decoder for special applications.
	IMWritePngStrategyFixed = 4
)

// IMRead reads an image from a file into a Mat.
// The flags param is one of the IMReadFlag flags.
// If the image cannot be read (because of missing file, improper permissions,
// unsupported or invalid format), the function returns an empty Mat.
//
// For further details, please see:
// http://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56
//
func IMRead(name string, flags IMReadFlag) Mat {
	cName := C.CString(name)
	defer C.free(unsafe.Pointer(cName))

	return newMat(C.Image_IMRead(cName, C.int(flags)))
}

[esimov]

Once the object rotation feature will be implemented this won't be an issue anymore.

[esimov]

With the new release of Pigo, it's possible now to detect rotated faces however with a small limitation, since you have to provide a specific angle to match the faces against it. Here is an example to detect the 4th missed face undetected previously:

$ pigo -in ~/Desktop/49496258-66536f80-f8a0-11e8-965b-4bdfb7f14524.jpg -out ~/Desktop/output.jpg -cf data/facefinder -angle=0.5 -iou=0.2

But this means to detect all the faces in an image the same command should be ran with the angle parameter ranging from 0.0 to 1.0, but this will cost in performance. Maybe in a feature release i will focus on resolving this kind of limitation.

[nathany]

Hey Endre. Thanks for the library. I too am wondering about detecting faces that are in a profile view.

I tested both eunseo.jpg and jimin3.jpg from this dataset https://github.com/Kagami/go-face-testdata without specifying an angle, but pigo didn't find a face.

[esimov]

Hey Nathan, that's true. I will try to investigate and get a workaround for this kind of limitation. Anyway i'm not really sure if the detector can detect 100% profile faces, but can detect deviations from face views.

[nathany]

Okay.

[olekukonko]

@esimov any update on this?

[esimov]

Not yet, right now i'm working on WASM support. Afterwards i can check this issue.

[codenoid]

how it's going

[kafan1986]

@esimov Any idea whether it will be possible to support profile or semi-profile face images?

[esimov]

Any idea whether it will be possible to support profile or semi-profile face images?

Not yet!

[ItsJasonPan]

I feel like this should be the top priority for now. Without rotation detection, performance on video streams is horrible even with the slightest movement of the head.

[esimov]

I have a few ideas in my mind how this issue could be resolved, but meantime any contribution is welcomed ;).