Fast face detection, pupil/eyes localization and facial landmark points detection library in pure Go.

Overview

pigo-logo

Build Status go.dev reference license release snapcraft

Pigo is a pure Go face detection, pupil/eyes localization and facial landmark points detection library based on Pixel Intensity Comparison-based Object detection paper (https://arxiv.org/pdf/1305.4537.pdf).

Rectangle face marker Circle face marker
rectangle circle

Motivation

The reason why Pigo has been developed is because almost all of the currently existing solutions for face detection in the Go ecosystem are purely bindings to some C/C++ libraries like OpenCV or dlib, but calling a C program trough cgo introduces huge latencies and implies a significant trade-off in terms of performance. Also in many cases installing OpenCV on various platforms is cumbersome.

The Pigo library does not require any additional modules or third party applications to be installed, however in case you wish to run the library in a real time, webcam based desktop application you might need to have Python and OpenCV installed. Head over to this subtopic for more explanation.

Key features

  • Does not require OpenCV or any 3rd party modules to be installed
  • High processing speed
  • There is no need for image preprocessing prior detection
  • There is no need for the computation of integral images, image pyramid, HOG pyramid or any other similar data structure
  • The face detection is based on pixel intensity comparison encoded in the binary file tree structure
  • Fast detection of in-plane rotated faces
  • The library can detect even faces with eyeglasses
  • Pupils/eyes localization
  • Facial landmark points detection
  • Webassembly support πŸŽ‰

Todo

  • Object detection and description

The library can also detect in plane rotated faces. For this reason a new -angle parameter has been included into the command line utility. The command below will generate the following result (see the table below for all the supported options).

$ pigo -in input.jpg -out output.jpg -cf cascade/facefinder -angle=0.8 -iou=0.01
Input file Output file
input output

Note: In case of in plane rotated faces the angle value should be adapted to the provided image.

Pupils / eyes localization

Starting from v1.2.0 Pigo offers pupils/eyes localization capabilities. The implementation is based on Eye pupil localization with an ensemble of randomized trees.

Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/master/examples/puploc

puploc

Facial landmark points detection

v1.3.0 marks a new milestone in the library evolution, Pigo being able for facial landmark points detection. The implementation is based on Fast Localization of Facial Landmark Points.

Check out this example for a realtime demo: https://github.com/esimov/pigo/tree/master/examples/facial_landmark

flp_example

Install

Install Go, set your GOPATH, and make sure $GOPATH/bin is on your PATH.

$ export GOPATH="$HOME/go"
$ export PATH="$PATH:$GOPATH/bin"

Next download the project and build the binary file.

$ go get -u -f github.com/esimov/pigo/cmd/pigo
$ go install

Binary releases

In case you do not have installed or do not wish to install Go, you can obtain the binary file from the releases folder.

The library can be accessed as a snapcraft function too.

snapcraft pigo

API

Below is a minimal example of using the face detection API.

First you need to load and parse the binary classifier, then convert the image to grayscale mode, and finally to run the cascade function which returns a slice containing the row, column, scale and the detection score.

cascadeFile, err := ioutil.ReadFile("/path/to/cascade/file")
if err != nil {
	log.Fatalf("Error reading the cascade file: %v", err)
}

src, err := pigo.GetImage("/path/to/image")
if err != nil {
	log.Fatalf("Cannot open the image file: %v", err)
}

pixels := pigo.RgbToGrayscale(src)
cols, rows := src.Bounds().Max.X, src.Bounds().Max.Y

cParams := pigo.CascadeParams{
	MinSize:     20,
	MaxSize:     1000,
	ShiftFactor: 0.1,
	ScaleFactor: 1.1,

	ImageParams: pigo.ImageParams{
		Pixels: pixels,
		Rows:   rows,
		Cols:   cols,
		Dim:    cols,
	},
}

pigo := pigo.NewPigo()
// Unpack the binary file. This will return the number of cascade trees,
// the tree depth, the threshold and the prediction from tree's leaf nodes.
classifier, err := pigo.Unpack(cascadeFile)
if err != nil {
	log.Fatalf("Error reading the cascade file: %s", err)
}

angle := 0.0 // cascade rotation angle. 0.0 is 0 radians and 1.0 is 2*pi radians

// Run the classifier over the obtained leaf nodes and return the detection results.
// The result contains quadruplets representing the row, column, scale and detection score.
dets := classifier.RunCascade(cParams, angle)

// Calculate the intersection over union (IoU) of two clusters.
dets = classifier.ClusterDetections(dets, 0.2)

A note about imports: in order to decode the generated image you have to import image/jpeg or image/png (depending on the provided image type) as in the following example, otherwise you will get a "Image: Unknown format" error.

import (
    _ "image/jpeg"
    pigo "github.com/esimov/pigo/core"
)

Usage

A command line utility is bundled into the library.

$ pigo -in input.jpg -out out.jpg -cf cascade/facefinder

Supported flags:

$ pigo --help

β”Œβ”€β”β”¬β”Œβ”€β”β”Œβ”€β”
β”œβ”€β”˜β”‚β”‚ ┬│ β”‚
β”΄  β”΄β””β”€β”˜β””β”€β”˜

Go (Golang) Face detection library.
    Version: 1.4.2

  -angle float
    	0.0 is 0 radians and 1.0 is 2*pi radians
  -cf string
    	Cascade binary file
  -flpc string
    	Facial landmark points cascade directory
  -in string
    	Source image (default "-")
  -iou float
    	Intersection over union (IoU) threshold (default 0.2)
  -json string
    	Output the detection points into a json file
  -mark
    	Mark detected eyes (default true)
  -marker string
    	Detection marker: rect|circle|ellipse (default "rect")
  -max int
    	Maximum size of face (default 1000)
  -min int
    	Minimum size of face (default 20)
  -out string
    	Destination image (default "-")
  -plc string
    	Pupils/eyes localization cascade file
  -scale float
    	Scale detection window by percentage (default 1.1)
  -shift float
    	Shift detection window by percentage (default 0.1)

Important notice: In case you wish to run also the pupil/eyes localization, then you need to use the plc flag and provide a valid path to the pupil localization cascade file. The same applies for facial landmark points detection, only that this time the parameter accepted by the flpc flag is a directory to the facial landmark point cascade files found under cascades/lps.

CLI command examples

You can also use the stdin and stdout pipe commands:

$ cat input/source.jpg | pigo > -in - -out - >out.jpg -cf=/path/to/cascade

in and out default to - so you can also use:

$ cat input/source.jpg | pigo >out.jpg -cf=/path/to/cascade
$ pigo -out out.jpg < input/source.jpg -cf=/path/to/cascade

Using the empty string as value for the -out flag will skip the image generation part. This, combined with the -json flag will encode the detection results into the specified json file. You can also use the pipe - value combined with the -json flag to output the detection coordinates to the standard (stdout) output.

Real time face detection (running as a shared object)

In case you wish to test the library real time face detection capabilities, the examples folder contains a few demos written in Python.

But why Python you might ask? Because the Go ecosystem is (still) missing a cross platform and system independent library for accessing the webcam.

In the Python program we are accessing the webcam and transferring the pixel data as a byte array through cgo as a shared object to the Go program where the core face detection is happening. But as you can imagine this operation is not cost effective, resulting in lower frame rates than the library is capable of.

These demos were created before the Webassembly port and were kept for showing the way how to integrate the Pigo library into other programming languages.

WASM (Webassembly) support πŸŽ‰

Important note: in order to run the Webassembly demos at least Go 1.13 is required!

Starting from version v1.4.0 the library has been ported to WASM. This gives the library a huge performance gain in terms of real time face detection capabilities.

WASM demo

To run the wasm demo select the wasm folder and type make.

For more details check the subpage description: https://github.com/esimov/pigo/tree/master/wasm.

Benchmark results

Below are the benchmark results obtained running Pigo against GoCV using the same conditions.

    BenchmarkGoCV-4   	       3	 414122553 ns/op	     704 B/op	       1 allocs/op
    BenchmarkPIGO-4   	      10	 173664832 ns/op	       0 B/op	       0 allocs/op
    PASS
    ok  	github.com/esimov/gocv-test	4.530s

The code used for the above test can be found under the following link: https://github.com/esimov/pigo-gocv-benchmark

Author

License

Copyright Β© 2019 Endre Simo

This software is distributed under the MIT license. See the LICENSE file for the full license text.

Issues
  • Comparison of two face

    Comparison of two face

    I use a stream of images and one of the tasks is to group faces, could you recommend a method for comparing faces.

    opened by deepch 16
  • oval shape

    oval shape

    Is it possible to use oval shape instead of square or circle?

    enhancement 
    opened by pjebs 15
  • Eye/Facial Coordinates in JSON?

    Eye/Facial Coordinates in JSON?

    Hi there, is it possible to print/output the Facial Coordinates/Pupil Localization pointers in the JSON file with -json itself? Or any work around for the same?

    or any other way to print the exact coordinates of the pupils or the other facial features?

    enhancement 
    opened by rgvxlens 14
  • wasm - demo nor sample working

    wasm - demo nor sample working

    opened by reachkr12 14
  • Infinite loop

    Infinite loop

    This particular line is a possible (an in fact, experienced) infinite loop:

    https://github.com/esimov/pigo/blob/0110acf7753778761ddd7b2ac4659de1c65b9c4b/core/pigo.go#L246

    I've figured it out from the stack trace:

    github.com/esimov/pigo/core.(*Pigo).classifyRegion(0xc0001a40a0, 0x42, 0x36, 0x9, 0xc00068e000, 0x238c, 0x238c, 0x5b, 0xbf800000)
    

    The gorutine track trace parameters are as follows:

    • 0xc0001a40a0 (the receiver)
    • 0x42 row (constantly modified)
    • 0x36 col (constantly modified)
    • 0x9 scale (!!does not change!!)
    • other params (do not change)

    The issue comes from a rounding /casting error. The main readme (as well as my code) sets the cascade parameters scale factor as 1.1. When the scale is 9, the factor would come up with 9.9, which again becomes 9 because of the int() conversion.

    The particular issue is that the source image was 91x100 (don't even ask), and I've calculated the face min/max size as 10% and 50% of the image width respectively. The minimum face size thus becomes 9 and with the scale ratio of 1.1 = infinite loop.

    Possible fixes:

    • Clamp the minimum value for a face size where it wouldn't cause problems (10px)
    • Better main loop for this stuff to avoid rounding errors
    investigation 
    opened by titpetric 11
  • Question: Facial/Identity Recogntition possible?

    Question: Facial/Identity Recogntition possible?

    Would it be possible to extend this library or use it as-is to perform facial recognition and thus identity of persons in images? (as opposed to face detection)

    opened by prologic 10
  • Realtime + WASM + blinking

    Realtime + WASM + blinking

    I was wondering if this library can be used for detecting blinks.

    You have a real-time example using python. Since chrome now supports (experimental) shape detection and video camera access: https://paul.kinlan.me/face-detection/ & https://medium.com/@joomiguelcunha/lets-play-with-chrome-s-face-detection-api-ca13017a958f I was wondering if using your WASM port it was able to detect if a face was blinking.

    opened by pjebs 6
  • Passing output value via a variable (Python)

    Passing output value via a variable (Python)

    Hi there! I'm trying to pass the values for the eyes separately using the blinkdet example, but seems like it outputs the values for both the eyes together and not separately (just outputting max_radius)

    Is there a way to output the values that I get using -json via a particular variable or just output the values (max_radius) of both the eyes separately?

    opened by rgvxlens 5
  • Saved image actually can not be opened

    Saved image actually can not be opened

    Like this image link

    It processed but can not open

    opened by jinfagang 5
  • web example

    web example

    Hi,

    If I run your web example than I get this error: 2020/04/24 21:51:38 [ERROR] reading next part multipart: NextPart: EOF I must admit that I don't meet the requirements for this example. My python is python3 and my opencv is 4.2. OS is Ubuntu 20.04LTS. I tested as follows

    1. Run: go run main.go -cf "../../cascade/facefinder"
    2. Open URL: localhost:8081/cam in browser
    3. Then this error appears Any ideas? Thanks
    opened by gerben-b 4
  • encountered some problems when i try write a demo by myself

    encountered some problems when i try write a demo by myself

    my demo is : https://github.com/Taoja/web-wasm-faceDetector

    there have two problems:

    • the func rgbaToGrayscale is useless, the picture still colorful when i output imageData and use canvas render it
    • unable detect face, maybe it is because func rgbaToGrayscale is useless
    opened by Taoja 4
  • Likeliness between faces

    Likeliness between faces

    First off, just want to appreciate and thank you for this cool project you're working on. I was wondering if you could add the feature to compare two faces and describe how likely they are.

    enhancement 
    opened by RealHarshThakur 11
  • Bug on demo

    Bug on demo

    Hello, I had a bug when I execute demo. It's nothing bad ;) .

    github.com/esimov/pigo/examples/facedet #Β python2 demo.py Traceback (most recent call last): File "demo.py", line 66, in cv2.circle(frame, (int(det[1]), int(det[0])), int(det[2]/2.0), (0, 0, 255), 2) OverflowError: signed integer is greater than maximum

    opened by Archie1978 2
  • Detect rotated faces

    Detect rotated faces

    The number of faces in this image should be 4 rectangles, so why get -out file is 3 rectangles (-iou 0.1 or -iou 0.2) duogeren

    -iou 0.1

    outduogeren

    -iou 0.2 outduogeren

    opened by fengweiqiang 14
Releases(v1.4.4)
Owner
Endre Simo
Programmer, crafting things and ideas mostly around graphics, image processing, computer vision and machine learning.
Endre Simo
Library to use HTML5 Canvas from Go-WASM, with all drawing within go code

go-canvas go-canvas is a pure go+webassembly Library for efficiently drawing on a html5 canvas element within the browser from go without requiring ca

null 137 Jun 25, 2021
Vugu: A modern UI library for Go+WebAssembly (experimental)

Vugu: A modern UI library for Go+WebAssembly (experimental)

Vugu 4.1k Jul 22, 2021
πŸΉπŸ•ΈοΈ WebAssembly runtime for Go

Wasmer Go Website β€’ Docs β€’ Slack Channel A complete and mature WebAssembly runtime for Go based on Wasmer. Features Easy to use: The wasmer API mimics

Wasmer 1.5k Jul 23, 2021
πŸΉπŸ•ΈοΈ WebAssembly runtime for Go

Wasmer Go Website β€’ Docs β€’ Slack Channel A complete and mature WebAssembly runtime for Go based on Wasmer. Features Easy to use: The wasmer API mimics

Wasmer 1.5k Jul 23, 2021
DOM library for Go and WASM

Go DOM binding (and more) for WebAssembly This library provides a Go API for different Web APIs for WebAssembly target. It's in an active development,

Denys Smirnov 434 Jul 24, 2021
Go compiler for small places. Microcontrollers, WebAssembly, and command-line tools. Based on LLVM.

TinyGo - Go compiler for small places TinyGo is a Go compiler intended for use in small places such as microcontrollers, WebAssembly (Wasm), and comma

TinyGo 8.3k Jul 19, 2021
Golang-WASM provides a simple idiomatic, and comprehensive API and bindings for working with WebAssembly for Go and JavaScript developers

A bridge and bindings for JS DOM API with Go WebAssembly. Written by Team Ortix - Hamza Ali and Chan Wen Xu. GOOS=js GOARCH=wasm go get -u github.com/

TeamOrtix 43 Jul 19, 2021
βš™οΈ Concept of Golang HTML render engine with frontend components and dynamic behavior

An HTML render engine concept that brings frontend-like components experience to the server side with native html/template on steroids. Supports any s

Yurii Zinets 57 Jul 16, 2021
Interact with browser from Go. Manually-crafted WebAPI interoperation library.

GWeb: golang + js + wasm gweb -- strictly typed WebAPI library on top of syscall/js. Like flow or TypeScript but for Go. You need it if you want to in

Life4 166 Jul 4, 2021
Framework for building distributed services with Web Assembly

Tarmac Framework for building distributed services with Web Assembly Tarmac is a unique framework designed for the next generation of distributed syst

Benjamin Cane 23 Jul 22, 2021
A Go unikernel running on x86 bare metal

EggOS A Go unikernel running on x86 bare metal Run a single Go applications on x86 bare metal, written entirely in Go (only a small amount of C and so

fanbingxin 1.3k Jul 20, 2021
Go Lang Web Assembly bindings for DOM, HTML etc

WebAPI Go Language Web Assembly bindings for DOM, HTML etc WARNING: The current API is in very early state and should be consider to be expremental. T

Go Web API 86 Jul 15, 2021
WebAssembly interop between Go and JS values.

vert Package vert provides WebAssembly interop between Go and JS values. Install GOOS=js GOARCH=wasm go get github.com/norunners/vert Examples Hello W

null 50 Jul 16, 2021
Autopilot in Go for docking the SpaceX Dragon capsule in the simulator

Dragon ISS Docking Autopilot in Go Autopilot written in Go and executed as WebAssembly for docking the SpaceX Dragon capsule in the official simulator

Martin Bertschler 288 Jul 21, 2021