HackIllinois 2019

_HackIllinois_2019.md

UPDATE: I'm in room 2017 on ECEB

Mort and OpenCV, Go, Python, Oh My Fish

Mentor: Pablo Aguiar (@scorphus)

I'm very happy to present the ideas/projects that I bring to HackIllinois 2019! They are:

• Mort and OpenCV
• Go
• Python
• Oh My Fish

Mort and OpenCV

This project consists of adding face recognition and feature detection support to Mort with OpenCV.

Mort is a storage and image processing server written in Go. Check its features.

OpenCV (Open Source Computer Vision Library) is an open-source BSD-licensed library that includes several hundreds of computer vision algorithms. Check the next markdown for more.

Go

Another project we can work on, improving docs, adding more examples, solving bugs and even, who knows, proposing new features. Go has a Contribution Guide and a very nice How to Contribute to Go. Beginner and seasoned contributors are more than welcome!

Python

We can also work on Python, by improving docs, adding examples, solving bugs, etc. You don't need to be an expert nor know C to contribute, as it's more than 60% written in... Python! The developer’s guide has information on how to contribute. It's awesome to contribute to Python! Beginner and seasoned contributors are more than welcome!

Oh My Fish

Last but not least, I also bring Oh My Fish to HackIllinois! It is a Fish Shell framework that provides core infrastructure to allow you to install packages which extend or modify the functionalities and look of your shell. It's fast, extensible and easy to use. There are some issues to solve and also would be really cool if you have an idea for a new plugin! You can also create a new theme.

_Mort_and_OpenCV.md

More on Mort and OpenCV

For this we'll need the following:

• Go: Install and check your installation
• Mort: Storage and image processing server written in Go
• GoCV: Computer Vision and OpenCV 3. Install locally (linux/macos/windows/) or use a Docker image (check below)
• OpenCV: Open Source Computer Vision Library. As the links above already install OpenCV 3.4, we'll be mostly interested in the docs and the trained classifiers for detecting objects

GoCV

We're better using Docker images to build and run GoCV applications.

1. Build the image according to the Dockerfile below.

docker build . -t hackillinois/gocv:4.0.1-buildstage

2. Run it with a mount:

docker run --rm -ti \
       --mount type=bind,source="$(pwd)",destination=/data \
       --entrypoint='' \
       hackillinois/gocv:4.0.1-buildstage bash

4. You should have a Docker image with GoCV and Mort. We can continue from here.

Face recognition and feature detection

Below, there's an initial prototype of face recognition and feature detection in Go using GoCV.

Once faces are recognized we can determine a point of interest, which cropping operations will take into consideration. In order properly determine the interest point we need to find the center of mass for all the faces recognized. For that we need center and weight of each rectangle. We define weight in this scenario as the area of the rectangle.

Consider faces found at 10x10, 150x100, 300x300. In this example (x, y being the coordinates of the rectangle's center and z the rectangle weight):

Face 1 - (x = (10 + 100) / 2 = 55), (y = (10 + 100) / 2 = 55), (z = 100 * 100 = 10000)

Face 2 - (x = (150 + 100) / 2 = 125), (y = (100 + 100) / 2 = 100), (z = 100 * 100 = 10000)

Face 3 - (x = (300 + 80) / 2 = 190), (y = (300 + 50) / 2 = 175), (z = 80 * 50 = 4000)

In order to find the center of mass we'll do a weighted average of the X and Y coordinates of the faces using:

Horizontal Axis - X (((55 * 10000) + (125 * 10000) + (190 * 4000)) / 24000 = 106)

Vertical Axis - Y (((55 * 10000) + (100 * 10000) + (175 * 4000)) / 24000 = 93).

So for the faces found by OpenCV in that image we have the center of mass of the picture being 106x93.

Dockerfile

FROM denismakogon/gocv-alpine:4.0.1-buildstage

RUN apk update && apk add brotli-dev expat-dev

RUN cd && \
    curl -L https://github.com/libvips/libvips/releases/download/v8.7.4/vips-8.7.4.tar.gz | tar xz && \
    cd vips-8.7.4 && ./configure && make && make install && \
    curl -L https://raw.githubusercontent.com/golang/dep/master/install.sh | sh && \
    go get -u -d gocv.io/x/gocv && \
    go get github.com/aldor007/mort/cmd/mort

ENV CGO_CFLAGS_ALLOW "-Xpreprocessor"

facefeat.go

package main

import (
	"fmt"
	"image"
	"image/color"
	"io/ioutil"
	"os"
	"path"

	"gocv.io/x/gocv"
)

const (
	frontalFaceAlt     = "haarcascade_frontalface_alt.xml"
	frontalFaceDefault = "haarcascade_frontalface_default.xml"
	profileFace        = "haarcascade_profileface.xml"
)

type Detector struct {
	ccFaceAlt     gocv.CascadeClassifier
	ccFaceDefault gocv.CascadeClassifier
	ccProfileFace gocv.CascadeClassifier
}

func (d *Detector) cleanUp() {
	d.ccFaceAlt.Close()
	d.ccFaceDefault.Close()
	d.ccProfileFace.Close()
}

func (d *Detector) prepare(ccDir string) error {
	d.ccFaceAlt = gocv.NewCascadeClassifier()
	if !d.ccFaceAlt.Load(path.Join(ccDir, frontalFaceAlt)) {
		return fmt.Errorf("Error reading cascade classifier file: %v", frontalFaceAlt)
	}
	d.ccFaceDefault = gocv.NewCascadeClassifier()
	if !d.ccFaceDefault.Load(path.Join(ccDir, frontalFaceDefault)) {
		return fmt.Errorf("Error reading cascade classifier file: %v", frontalFaceDefault)
	}
	d.ccProfileFace = gocv.NewCascadeClassifier()
	if !d.ccProfileFace.Load(path.Join(ccDir, profileFace)) {
		return fmt.Errorf("Error reading cascade classifier file: %v", profileFace)
	}
	return nil
}

func (d *Detector) loadImage(inFileName string) (*gocv.Mat, error) {
	img := gocv.IMRead(inFileName, gocv.IMReadColor)
	if img.Empty() {
		return nil, fmt.Errorf("image is empty")
	}
	return &img, nil
}

func (d *Detector) detectMultiScale(img *gocv.Mat) ([]image.Rectangle, color.RGBA) {
	if rects := d.ccFaceAlt.DetectMultiScale(*img); len(rects) > 0 {
		fmt.Print(" [ccFaceAlt]")
		return rects, color.RGBA{0, 255, 0, 0}
	}
	if rects := d.ccFaceDefault.DetectMultiScaleWithParams(*img, 1.15, 4, 0, image.Pt(0, 0), image.Pt(0, 0)); len(rects) > 0 {
		fmt.Print(" [ccFaceDefault]")
		return rects, color.RGBA{0, 0, 255, 0}
	}
	if rects := d.ccProfileFace.DetectMultiScaleWithParams(*img, 1.2, 4, 0, image.Pt(0, 0), image.Pt(0, 0)); len(rects) > 0 {
		fmt.Print(" [ccProfileFace]")
		return rects, color.RGBA{255, 0, 0, 0}
	}
	return nil, color.RGBA{}
}

func drawRects(img *gocv.Mat, rects []image.Rectangle, color color.RGBA) {
	for _, r := range rects {
		gocv.Rectangle(img, r, color, 2)
	}
}

func (d *Detector) detectFaces(img *gocv.Mat) bool {
	rects, color := d.detectMultiScale(img)
	if rects == nil {
		return false
	}
	drawRects(img, rects, color)
	return true
}

func (d *Detector) detectFeatures(img *gocv.Mat) bool {
	imgGray := gocv.NewMat()
	defer imgGray.Close()
	corners := gocv.NewMat()
	defer corners.Close()
	gocv.CvtColor(*img, &imgGray, gocv.ColorBGRAToGray)
	gocv.GoodFeaturesToTrack(imgGray, &corners, 20, 0.04, 1)
	if corners.Empty() {
		return false
	}
	for row := 0; row < corners.Rows(); row++ {
		corner := corners.GetVecfAt(row, 0)
		x, y := int(corner[0]), int(corner[1])
		gocv.Circle(img, image.Pt(x, y), 5, color.RGBA{0, 255, 0, 0}, 2)
	}
	return true
}

func (d *Detector) loadAndDetect(inFileName, outFileName string) error {
	img, err := d.loadImage(inFileName)
	if err != nil {
		return err
	}
	fmt.Printf(" <-- %s", inFileName)
	defer img.Close()
	if !d.detectFaces(img) && !d.detectFeatures(img) {
		return nil
	}
	if !gocv.IMWrite(outFileName, *img) {
		return fmt.Errorf("Error writing image to: %v", outFileName)
	}
	fmt.Printf(" %s -->\n", outFileName)
	return nil
}

func doit(ccDir, sourceDir, destinationDir string) error {
	detector := Detector{}
	if err := detector.prepare(ccDir); err != nil {
		return err
	}
	defer detector.cleanUp()
	files, err := ioutil.ReadDir(sourceDir)
	if err != nil {
		return err
	}
	for _, file := range files {
		if file.IsDir() {
			continue
		}
		inFileName := path.Join(sourceDir, file.Name())
		outFileName := path.Join(destinationDir, file.Name())
		if err := detector.loadAndDetect(inFileName, outFileName); err != nil {
			fmt.Printf("Failed to load %s: %s\n", inFileName, err)
		}
	}
	return nil
}

func main() {
	if len(os.Args) < 4 {
		fmt.Println("How to run:\n\tshowimage <classifiers_dir> <source_dir> <destination_dir>")
		return
	}
	ccDir := os.Args[1]
	sourceDir := os.Args[2]
	destinationDir := os.Args[3]
	fmt.Printf("Using %s on pics in %s saving in %s\n", ccDir, sourceDir, destinationDir)
	if err := doit(ccDir, sourceDir, destinationDir); err != nil {
		fmt.Println(err)
	}
}

presentation.pdf

1. Build a Docker image from the Dockerfile below

docker build . -t hackillinois/gocv

FROM denismakogon/gocv-alpine:4.0.1-buildstage
RUN apk update; apk add brotli-dev expat-dev
RUN cd; \
    curl -L https://github.com/libvips/libvips/releases/download/v8.7.4/vips-8.7.4.tar.gz | tar xz; \
    cd vips-8.7.4; ./configure; make; make install

2. Run it with a mount:

docker run --rm -ti \
       --mount type=bind,source="$(pwd)",destination=/data \
       --entrypoint='' \
       hackillinois/gocv bash