Hour Of Code - JavaScript AI Face Detection

JavaScript AI Face Detection 1-hour workshop for Hour Of Code HK

Let’s type real codes!

V2 / 20200827

1

Rundown: Who are we?

5

Concepts before we start

5

Introduction to P5JS Platform

10

0 - P5JS Basics

5

1 - Load Webcam

10

2 - Detect Face!

10

3 - Draw Emoji

10

Q&A 2

Who are we?

3

Background: Ms. Monica Leung ● Founder of Blueinno Technology Limited, STEAM Education Service Company Founded in 2016 ● 15+ years experience of Hardware & Software Development & Project management. ● MSc Electronic Engineering (CUHK) ● BEng Electronic Engineering (Major), Computer Science (Minor) ● Worked in Various Software & Hardware Companies or startup, including ASTRI, Motorola, Vtech, etc.

4

Background: Mr. Calvin Kam ● Lead Instructor & Software Engineer of Blueinno Technology Limited ● 5+ years experience of Software Development ● PhD Candidate, Computer Science and Engineering, CUHK ● BSc (1st Hons) in Computer Science, HKBU ● Research in Computer Vision & AI

5

AI Entrepreneurship Program (since 2018)

Today’s Project: Face Detection

Let’s type real codes!

7

What face can do?

Animoji

Drowsiness Detection

Facial features

Face Recognition for surveillance

8

Face Detection vs Face Recognition

Which one is Face Detection? Which one is Face Recognition?

9

Today’s Project

10

Concepts: X-Y Coordinates

11

In maths, we have X- Y Coordinates

X Y Coordinates

12

In Scratch, we try to put the cat to a speciďŹ c location with (X, Y)

13

How about P5JS?

In P5.JS, we start our origin at the top left corner

(0, 0) X=0 Y= 0

14

Coordinates To draw things, we must tell the computer how to draw.

Start drawing here (starting point)

Draw 6 squares to the right (width = 6)

Draw 7 squares to the down (height = 7)

15

CONCEPTS:

Question 0 How can we draw this red rectangle?

2

10

1

Starting Point: ____________ Width: ___________ Height: __________

8

16

CONCEPTS:

Question 0 How can we draw this red rectangle?

2

10

1

(2,1)

Starting Point: ____________

8

Width: ___________

7 Height: __________

8

17

CONCEPTS:

Question 0

1

10

How can we draw this red rectangle?

(1,4)

Starting Point: ____________

4

3 Width: ___________ Height: __________ 4

8

18

CONCEPTS:

Question 0

1

10

How can we draw this red rectangle?

(1,4)

Starting Point: ____________

4

3 Width: ___________ Height: __________ 4

8

19

Prerequisites 1. Prepare your computer with WebCam (Windows / Mac ) 2. With Chrome browser installed 3. Passion to learn coding!

20

Introduction to P5JS Platform

21

Processing Family

Slightly different in terms of syntax(Grammar) Processing 3 (Java)

p5.js (Javascript)

22

We will use P5js to code ● Javascript! ● Easy to draw graphics ● Easy to work with deep learning libraries!

23

What can be done on P5.JS ?

Games

Arts

AI 24

Run first, and once only

Main Structure void setup() {

void setup() {

=> }

}

void loop() {

void draw() {

}

}

Arduino

Processing

25

Run forever

Main Structure void setup() {

void setup() {

}

}

void loop() {

void draw() {

}

}

=> Arduino

Processing

26

P5Js Function function preload() { Run First, Optional !!

}

function setup() { Run Once

} function draw() {

Run forever!

} 27

Sequence of Processing Program

0 void preload()

void setup()

void draw()

28

P5Js Function Questions?

What is the correct running sequence?

A.) Preload → Draw → Setup

B.) Setup → Preload → Draw

C.) Draw → Setup → Preload

D.) Preload → Setup → Draw

29

P5Js Function Questions?

What is the correct running sequence?

A.) Preload → Draw → Setup

B.) Setup → Preload → Draw

C.) Draw → Setup → Preload

D.) Preload → Setup → Draw

30

Checking Manual (API) https://p5js.org/

31

Check the usage in the “Reference” page ! https://p5js.org/

32

Now try your ďŹ rst program using p5.js!

33

https://editor.p5js.org/ Sign in with your Google Account

34

After Login, Go to http://bit.ly/hocblueinno

35

http://bit.ly/hocblueinno File → Duplicate for saving a copy

36

P5JS Basics

37

https://editor.p5js.org/

Coding Output

Set the editor theme

Console 38

39

Press “Play” See what’s happened

40

CONCEPTS:

JavaScript Syntax: Brackets function setup() { createCanvas(640, 480); }

41

CONCEPTS:

JavaScript Syntax: Brackets function setup() { createCanvas(640, 480); }

42

Let’s modify the background and see :)

function setup() { createCanvas(640, 480); } function draw() { background(220); }

Set the background colour background(g)

Try to modify the values!

43

background(10);

background(220);

background(100);

44

Project Today!

45

1. Load Webcam 2. Detect Face 3. Draw Emoji 46

1 2 3 4 5 6 7 8 9 10

let capture; function setup() { createCanvas(640, 480); } function draw() { background(220); }

47

1 2 3 4 5 6 7 8 9 10

let capture; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); } function draw() { background(220); }

48

1 2 3 4 5 6 7 8 9 10

let capture; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); } function draw() { background(220); image(capture, 0, 0); }

49

1 2 3 4 5 6 7 8 9 10 11

let capture; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); } function draw() { background(220); image(capture, 0, 0); }

50

Check the reference https://p5js.org/reference/

51

1. Load Webcam 2. Detect Face 3. Draw Emoji 52

Detect Face i) import script to use ml5 library ii) Load Model ii) Detect Face

53

What is ML5?

54

HTML Syntax

CONCEPTS:

Go to index.html

55

Heading, Body

56

Body

57

Script for ML5 library <script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>

58

Detect Face i) import script ii) Load Model ii) Detect Face

59

1 2 3 4 5 6 7 8 9 10 11 12 13 14

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function draw() { background(220); image(capture, 0, 0); } 60

61

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { } function draw() { background(220); image(capture, 0, 0); } 62

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); } function draw() { background(220); image(capture, 0, 0); } 63

Check the console

64

Detect Face i) import script ii) Load Model ii) Detect Face

65

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); } function draw() { background(220); image(capture, 0, 0); }

66

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); }

Change draw to gotResult

function gotResult(err, result) { background(220); image(capture, 0, 0); }

67

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); } function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { } }

68

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); }

Print only once in the console

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); } } 69

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

let capture; let faceapi; function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); }

Detect repeatly

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); } faceapi.detectSingle(gotResult); }

70

More objects print out

71

Expand & see

72

1. Load Webcam 2. Detect Face 3. Draw Emoji 73

Target: Draw a rectangle for the face detected

74

1 2 3 4 5 6 7 8 9 10 11 12 13 14

let capture; let faceapi; let img; function preload() { img = loadImage("emoji.png"); }

Image must be preloaded

function setup() { createCanvas(640, 480); capture = createCapture(VIDEO); capture.hide(); faceapi = ml5.faceApi(capture, modelReady); }

75

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); let face = result.alignedRect._box; let x = face._x; let y = face._y; let w = face._width; let h = face._height; rect(x, y, w, h); image(img,x,y,w,h); } faceapi.detectSingle(gotResult); }

Get the face dimension box

76

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); let face = result.alignedRect._box; let x = face._x; let y = face._y; let w = face._width; let h = face._height; rect(x, y, w, h); image(img,x,y,w,h); } faceapi.detectSingle(gotResult); }

Get the x, y, width and height

77

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); let face = result.alignedRect._box; let x = face._x; let y = face._y; let w = face._width; let h = face._height; rect(x, y, w, h); image(img,x,y,w,h); } faceapi.detectSingle(gotResult); }

Draw the rectangle

78

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

function gotResult(err, result) { background(220); image(capture, 0, 0); if(result) { print(result); let face = result.alignedRect._box; let x = face._x; let y = face._y; let w = face._width; let h = face._height; rect(x, y, w, h); image(img,x,y,w,h); } faceapi.detectSingle(gotResult); }

Draw the emoji

79

Full Codes let faceapi; let img; let video;

http://bit.ly/hocblueinno-face

function preload() { /* Preload the images to the variable */ img = loadImage("emoji.png"); } function setup() { /* Run once in the beginning */ createCanvas(640, 480); video = createCapture(VIDEO); video.hide(); faceapi = ml5.faceApi(video, modelReady); } function modelReady() { print("OK"); faceapi.detectSingle(gotResult); } function gotResult(err, result) { background(220); image(video, 0, 0); if (result) { print(result); let face = result.alignedRect._box; let x = face._x; let y = face._y; let w = face._width; let h = face._height; rect(x, y, w, h); image(img,x,y,w,h); } faceapi.detectSingle(gotResult); }

80

Contact us Email: teaching@blueinnotechnology.com : Blueinno Technology

81