Practical work "colors" KNN algorithm

K NEAREST NEIGHBORS ALGORITHM

PREREQUISITE KNOWLEDGE PYTHON • • • • • • MATHEMATICS 𝑎(𝑥𝑎 , 𝑦𝑎 ) 𝑒𝑡 𝑏(𝑥𝑏 , 𝑦𝑏 ) 𝑑 = √(𝑥𝑏 − 𝑥𝑎 )2 + (𝑦𝑏 − 𝑦𝑎 )2

SOFTWARE

HARDWARE • • •

OBJECTIVE

INTRODUCTION MACHINE LEARNING AND SUPERVISED LEARNING

• • •

𝑌≈ 𝐹(𝑋)

STUDY SITUATION ET TRAINING SET OF THE ALPHAI ROBOT

•

•

•

from PIL import Image import numpy as np import matplotlib.pyplot as plt def read_image(name): return np.array(Image.open(name, "r"))

img1=read_image("forward_0.jpg ") plt.imshow(img1) plt.show()

# give the image path

• • •

def rgb2gray(rgb): return np.dot(rgb[...,:3], [0.299, 0.587, 0.144]) img2=rgb2gray(img1) plt.imshow(img2, cmap=”gray”)

def 𝑚𝑒𝑎𝑛(𝑖𝑚𝑔, 𝑥, 𝑦, ℎ𝑒𝑖𝑔ℎ𝑡, 𝑤𝑖𝑑𝑡ℎ)

def get_two_values(img, x1=10, y1=28, x2=42, y2=28, height=16, width=13): value1 = average(img, x1, y1, height, width) value2 = average(img, x2, y2, height, width) return np.array((value1, value2))

THE K NEAREST NEIGHBORS ALGORITHM, KNN

Start Algorithm Input data: • •

a dataset D, corresponding to the elaborated training set a function for calculating the distance between the two considered points

•

of coordinates [x1,y1] and [x2,y2].𝑑 = √(𝑥2 − 𝑥1 )2 + (𝑦2 − 𝑦1 )2 An integer k, representing the number of neighbors to be considered

For a new observation X of which we want to predict his decision Y Do: 1. Calculate all distances of this observation X with the other observations in dataset D 2. Retain the K observations of dataset D closest to X using the distance calculation function d 3. Take the values of Y of the k observations retained. Perform a regression and calculate the mean (or median) of Y retained 4. Return the value calculated in step 3 as the value that was predicted by KNN for observation X. End Algorithm

def 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑝𝑜𝑖𝑛𝑡1, 𝑝𝑜𝑖𝑛𝑡2)

ROBOT TRAINING PREPARATION PHASE OF THE WORK ENVIRONMENT

→

→

→ →

TRAINING PHASE

•

• • •

TESTING PHASE

TO GO FURTHER:

PROGRAM THE K'S NEAREST NEIGHBORS ALGORITHM → →

→

→

PROGRAM THE ROBOT DIRECTLY

def 𝑡𝑎𝑘𝑒_𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛(𝑠𝑒𝑛𝑠𝑜𝑟𝑠: 𝑙𝑖𝑠𝑡) → 𝑖𝑛𝑡

ALGORITHM PROGRAMMING DISTANCE CALCULATION

def 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑎: 𝑙𝑖𝑠𝑡, 𝑏: 𝑙𝑖𝑠𝑡) → 𝑓𝑙𝑜𝑎𝑡 𝑎 = [𝑥𝑎 , 𝑦𝑎 ] 𝑒𝑡 𝑏 = [𝑥𝑏 , 𝑦𝑏 ]

𝑑 = √(𝑥𝑏 − 𝑥𝑎 )2 + (𝑦𝑏 − 𝑦𝑎 )2

import math

y = math.sqrt(x)

y = x**2

# compute distance a = [0, 0] b = [1, 2] print("distance", distance(a, b))

CALCULATION OF ALL DISTANCES

def 𝑎𝑙𝑙_𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑠(𝑎: 𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡], 𝑡𝑟𝑎𝑖𝑛_𝑠𝑒𝑛𝑠𝑜𝑟𝑠: 𝑙𝑖𝑠𝑡[𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡]]) → 𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡] 𝑎 = [𝑥0 , 𝑦0 ]

𝑡𝑟𝑎𝑖𝑛_𝑠𝑒𝑛𝑠𝑜𝑟𝑠 = [[𝑥1 , 𝑦1 ], [𝑥2 , 𝑦2 ], … , [𝑥𝑛 , 𝑦𝑛 ]]

# compute all distances a = [0.4, 0.6] train_sensors = [[0, 0], [0, 1], [1, 0]] distance_list = all_distances(a, train_sensors) print('distances to data', distance_list)

FIND THE SMALLEST ELEMENT OF A TABLE

def 𝑓𝑖𝑛𝑑_𝑚𝑖𝑛𝑖𝑚𝑢𝑚(𝑑𝑙𝑖𝑠𝑡: 𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡]) → 𝑖𝑛𝑡

# minimum in a list idx_min = find_minimum(distance_list) print('index of minimum', idx_min)

NEAREST NEIGHBOR

def 𝑛𝑒𝑎𝑟𝑒𝑠𝑡_𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟_𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛(𝑡𝑟𝑎𝑖𝑛_𝑠𝑒𝑛𝑠𝑜𝑟𝑠: 𝑙𝑖𝑠𝑡[𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡]], 𝑡𝑟𝑎𝑖𝑛_𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑠: 𝑙𝑖𝑠𝑡[𝑖𝑛𝑡], 𝑎: 𝑙𝑖𝑠𝑡[𝑓𝑙𝑜𝑎𝑡]) → 𝑖𝑛𝑡

[𝑑1 , … , 𝑑𝑛 ]

𝑎 = [𝑥0 , 𝑦0 ] 𝑒𝑡 𝑡𝑟𝑎𝑖𝑛_𝑠𝑒𝑛𝑠𝑜𝑟𝑠 = [[𝑥1 , 𝑦1 ], [𝑥2 , 𝑦2 ], … , [𝑥𝑛 , 𝑦𝑛 ]] 𝑑𝑖 [𝑥𝑖 , 𝑦𝑖 ]

𝑡𝑟𝑎𝑖𝑛_𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑠 =

# KNN a = [0.4, 0.6] train_sensors = [[0, 0], [0, 1], [1, 0]] train_decisions = [1, 2, 0] decision = nearest_neighbor_decision(train_sensors, train_decisions, a) print('KNN', decision)

USING YOUR ALGORITHM WITH THE ROBOT

train_sensors = train_decisions = None def learn(X_train, y_train): global train_sensors, train_decisions train_sensors, train_decisions = X_train, y_train loss = 0 return loss def take_decision(sensors): if train_sensors is None : return 0 return nearest_neigbor_decision(train_sensors, train_decisions, sensors)

TO GO FURTHER K NEAREST NEIGHBORS

𝑘 > 1, 𝑘 ∈ ℕ

INTRODUCTION TO NEURAL NETWORKS

CONCLUSION :