HEALING CITY
2019 - 2020
Advisors: Angelos Chronis Nariddh Khean Serjoscha Duering
Student: Luyang Zhang
Program: MaCT02
BARCELONA
AI in Urbanism II
CONTEXT
PROJECT HEALING CITY
EMOTION
CONNECTION URBAN SCENE TYPOLOGY
URBAN ENVIRONMENT
RESIDENT
PROJECT RESEARCH MAP
URBAN SCENE TYPOLOGY
EMOTION
PROJECT PROJECT AIM
EMOTION
CONNECTION URBAN SCENE TYPOLOGY
URBAN ENVIRONMENT
RESIDENT
PROJECT
URBAN ENVIRONMENT
URBAN SCENE TYPOLOGY
AI CLASS - URBAN ENVIRONMENT ANALYSIS
HYPOTHESIS URBAN ENVIRONMENT
Is it possible to analyze the
URBAN ENVIRONMENT from a HUMAN PERSPECTIVE
OBJECTIVE STREET VIEW IMAGES
Is it possible to analyze the
URBAN ENVIRONMENT STREET VIEW from a IMAGES HUMAN PERSPECTIVE
RESEARCH QUESTION SPECIFIC QUESTION
How to make an image measurable?
How to measure the visual elements of urban environment that people perceive?
In order to analyze the urban scene typology.
AI
DATA MAGNITUDE
> 1000
MANUAL / AUTO
AUTO
DATA DIMENSION
MULTIDIMENSIONAL
DATA ANALYSIS
IMAGE PROCESSING
IMAGE
DATA GENERATION
DATA FORMAT
UNSUPERVISED LEARNING
USE OF AI-DRIVEN TECHNOLOGY
DATASET
DATASET DESCRIPTION GENERATE DATASET - PANORAMIC SEGMENTATION OF STREET VIEW IMAGES Through the panoramic segmentation (include instance segmentation and semantic segmentation) of street view images, the features used for urban environment analysis are obtained.
detect the
amount
STREET VIEW IMAGE
detect the
of each different object.
INSTANCE SEGMENTATION
PERSON
BICYCLE
VEHICLE
URBAN FURNITURE
ratio of different types of
physical environment area.
PANORAMIC SEGMENTATION
SEMANTIC SEGMENTATION
ROAD
SIDEWALK
VEGETATION
BUILDING
SKY
DATASET DESCRIPTION STREET VIEW IMAGES PANORAMIC SEGMENTATION RESULT Panoramic segmentation of 98,673 street view images, each image gets 14 segmentation features (include 6 instance segmentation features and 8 semantic segmentation features).
STREET VIEW IMAGE
98,673 STREET VIEW IMAGES
PERSON
BICYCLE
VEHICLE
URBAN_F
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
2
2
17
2
0.22
0.02
0.07
0.28
0.38
amount
%
14 FEATURES
DATASET DESCRIPTION SPATIAL DISTRIBUTION OF FEATURES PERSON
BICYCLE
VEHICLE
URBAN FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
BIRD
DOG
DATASET DESCRIPTION SPATIAL DISTRIBUTION OF FEATURES - CITY CENTRE
PERSON
Passeig de Gràcia La Sagrada Família
La Rambla
BICYCLE
VEHICLE
DATASET DESCRIPTION SPATIAL DISTRIBUTION OF FEATURES - CITY CENTRE
URBAN FURNITURE
ROAD
SIDEWALK
DATASET DESCRIPTION SPATIAL DISTRIBUTION OF FEATURES - CITY CENTRE
BUILDING
VEGETATION
SKY
DATASET DESCRIPTION SPATIAL DISTRIBUTION OF FEATURES - CITY CENTRE
BIRD
DOG
Parc del Clot Plaรงa de Catalunya
DATA PREPROCESSING SELECT FEATURES Initially select the features used for urban environment analysis.
STREET VIEW IMAGES PANORAMIC SEGMENTATION
non-numeric data
find the selected images by name
numeric data
visualization of spatial distribution
selected moving object features
selected stable physical environment
SELECTED FEATURES
DATA PREPROCESSING FIND MISSING DATA Check the dataset for missing data.
PERCENTAGE OF MISSING DATA FOR EACH FEATURE not missing
MISSING DATA HEATMAP
missing
PERSON - 0.0% BICYCLE - 0.0% VEHICLE - 0.0% URBAN_FURNITURE - 0.0% ROAD - 0.0% SIDEWALK - 0.0% BUILDING - 0.0% VEGETATION - 0.0% SKY - 0.0%
✌
NO MISSING DATA
DATA PREPROCESSING DATA DISTRIBUTION Check the distribution of data through the boxplot to determine how the data is scaled.
FEATURE HISTOGRAM
DATA PREPROCESSING DATA DISTRIBUTION Check the distribution of data through the boxplot to determine how the data is scaled.
PHYSICAL ENVIRONMENT FEATURE DISTRIBUTION
proportion
amount
MOVING OBJECT FEATURE DISTRIBUTION
SINCE EACH FEATURE HAS MANY OUTLIERS (THESE OUTLIERS ARE VALID INFORMATION) WHEN THR DATA IS SCALED, USING MinMaxScaler CAN SCALE SPARSE DATA BETTER
DATA PREPROCESSING STANDARDIZATION (MinMaxScaler) Eliminate the influence of different units and characteristic variable ranges on subsequent analysis.
SCALED FEATURE DISTRIBUTION
SCALE ALL THE DATA FROM 0 to 1
DATA ANALYSIS ALGORITHMS
DATA ANALYSIS PROPOSAL OF DATA ANALYSIS
PERSON
VEHICLE URBAN FURNITURE ROAD SIDEWALK BUILDING VEGETATION SKY
CLUSTER ANALYSIS
URBAN SCENE TYPOLOGY
BICYCLE
DATA ANALYSIS CLUSTERING ALGORITHM SELECTION
PERSON
VEHICLE
CLUSTER ANALYSIS KMEANS
URBAN FURNITURE
or
ROAD
HIERARCHICAL
SIDEWALK
or
BUILDING
KMEANS + HIERARCHICAL
VEGETATION SKY
URBAN SCENE TYPOLOGY
BICYCLE
DATA ANALYSIS CLUSTER ANALYSIS - KMEANS (unsupervised learning)
KMEANS CLUSTERING (K=40) PCA (dimensionality reduction) visualization
USE KMEANS FOR PRELIMINARY CLUSTERING AND OBTAIN 40 CLUSTERING CENTERS
FOR THR NEXT HIERARCHICAL CLUSTERING
DATA ANALYSIS CLUSTER ANALYSIS - HIERARCHICAL CLUSTERING (unsupervised learning)
HIERARCHICAL CLUSTERING
6 MAIN CLUSTER
19 SUB - CLUSTER
40 KMEANS CLUSTERING CENTERS
RESULT
DATA ANALYSIS URBAN SCENE TYPOLOGY SUMMARY
NARROW ALLEY or PEDESTRIAN AREA ON THE ROAD
17%
SQUARE or WIDE PEDESTRAIN STREET
8%
HIGHWAY or MAIN ROAD
20%
PEDESTRIAN AREA TRAFFIC - DRIVEN ROAD or or ROAD ROAD WITH PARKING WITH HIGH SPACE GREENING RATE
14%
26%
MIX STREET
15%
DATA ANALYSIS HIERARCHICAL CLUSTERING VS KMEANS CLUSTERING
KMEANS + HIERARCHICAL CLUSTERING (method='ward')
USE KMEANS TO DIRECTLY CLUSTER INTO SIX CLUSTERS
DATA ANALYSIS URBAN SCENE TYPOLOGY NARROW ALLEY PEDESTRIAN AREA ON THE ROAD 5%
VEHIC
AN RE B UR ITU RN U F
LE
ROA D
E
CL CY BI 7%
ALK W E ID
PERSON
17%
S
1%
Y
SK
G
VEGE
IN
N TATIO
ILD
BU
PERSON
BICYCLE
4%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY NARROW ALLEY PEDESTRIAN AREA ON THE ROAD
DATA ANALYSIS URBAN SCENE TYPOLOGY SQUARE WIDE PEDESTRAIN STREET 2%
VEHIC
N BA URE R U IT RN FU
LE
ROA D
E
CL CY BI 1%
ALK W E ID
PERSON
8%
S
1%
Y
SK
VEGE
NG
N TATIO
ILD I
BU
PERSON
BICYCLE
4%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY SQUARE WIDE PEDESTRAIN STREET
DATA ANALYSIS URBAN SCENE TYPOLOGY HIGHWAY MAIN ROAD 9%
VEHIC
N BA URE R U IT RN FU
LE
ROA D
E
CL CY BI
S
ALK W E ID
PERSON
20%
Y
SK
VEGE
NG
N TATIO
ILD I
BU
PERSON
BICYCLE
11%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY HIGHWAY MAIN ROAD
DATA ANALYSIS URBAN SCENE TYPOLOGY PEDESTRIAN AREA WITH HIGH GREENING RATE ROAD WITH HIGH GREENING RATE 7%
VEHIC
N BA URE R U IT RN FU
LE
ROA D
E
CL CY BI 2%
S
ALK W E ID
PERSON
14%
Y
SK
VEGE
NG
N TATIO
ILD I
BU
PERSON
BICYCLE
5%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY PEDESTRIAN AREA WITH HIGH GREENING RATE ROAD WITH HIGH GREENING RATE
DATA ANALYSIS URBAN SCENE TYPOLOGY TRAFFIC - DRIVEN ROAD ROAD WITH PARKING SPACE VEHIC
N BA URE R U IT RN FU
LE
15%
ROA D
E
CL CY BI 6%
S
ALK W E ID
PERSON
26%
Y
SK
VEGE
NG
N TATIO
ILD I
BU
PERSON
BICYCLE
5%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY TRAFFIC - DRIVEN ROAD ROAD WITH PARKING SPACE
DATA ANALYSIS URBAN SCENE TYPOLOGY MIX STREET
1%
VEHIC
N BA URE R U IT RN FU
LE
ROA D
E
CL CY BI 6%
S
ALK W E ID
PERSON
15%
Y
SK
VEGE
NG
N TATIO
ILD I
BU
PERSON
BICYCLE
8%
VEHICLE
URBAN_FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
DATA ANALYSIS URBAN SCENE TYPOLOGY MIX STREET
DATA ANALYSIS URBAN SCENE TYPOLOGY MAP NARROW ALLEY PEDESTRIAN AREA ON THE ROAD
SQUARE WIDE PEDESTRAIN STREET
HIGHWAY MAIN ROAD
HIGH GREENING RATE PEDESTRIAN AREA or ROAD
TRAFFIC - DRIVEN ROAD ROAD WITH PARKING SPACE
MIX STREET
DATA ANALYSIS NARROW ALLEY PEDESTRIAN AREA ON THE ROAD
SQUARE WIDE PEDESTRAIN STREET
HIGHWAY MAIN ROAD
HIGH GREENING RATE PEDESTRIAN AREA or ROAD
TRAFFIC - DRIVEN ROAD ROAD WITH PARKING SPACE
MIX STREET
Sant MartÃ
Ciutat Vella
URBAN SCENE TYPOLOGY MAP - ZOOM
DATA ANALYSIS
Sant MartÃ
Ciutat Vella
URBAN SCENE TYPOLOGY MAP - ZOOM
CONCLUSION
CONCLUSION HUMAN PERSPECTIVE URBAN ENVIRONMENT Is it possible to analyze the
URBAN ENVIRONMENT from a HUMAN PERSPECTIVE PANORAMIC SEGMENTATION
CLUSTER ANALYSIS
ANALYZE & EXPLAIN THE MEANING OF EACH CLUSTER NARROW ALLEY or PEDESTRIAN AREA ON THE ROAD
17%
SQUARE or WIDE PEDESTRAIN STREET
8%
HIGHWAY or MAIN ROAD
20%
PEDESTRIAN AREA TRAFFIC - DRIVEN ROAD or or ROAD ROAD WITH PARKING WITH HIGH SPACE GREENING RATE
14%
26%
MIX STREET
15%
KMEANS + HIERARCHICAL PERSON
BICYCLE
VEHICLE
URBAN FURNITURE
ROAD
SIDEWALK
BUILDING
VEGETATION
SKY
Through the analysis and clustering of street view images, the classification and distribution of urban scenes with human perspective can be better obtained.