Jieyu Computational Design Portfolio

Portfolio

Design as Democracy Selected Works 2016-2020

Jieyu Zhou Computational Design

Contents Democracy in the design process has always been debated among various

stakeholders. Participatory design is a hands-on action that inspires collective creativity, challenges environmental injustice, and increases the sense of inclusion. Approaches for realizing participatory design are complex and multidimensional, from the urban scale and community scale, to the human scale; from simply informing, to designing and constructing. My passion and goals lie in the intersection of design, urban data, and computational programming. I like to think that these goals share a common thread: using interaction technologies, big data, and deep-learning to facilitate (computationally) understanding, expressing, and engaging during the design process to achieve consensus for all sectors, which are my objectives for graduate studies.

Long-Term Projects //01 Community Co-Generation Platform Rapid scheme generation and evaluation [Deep learning GAN, Grasshopper]

//02 Mechanical Rockery

Traditional rockery simulation and fabrication [Unity 3D, Reinforce Learning, Robotic Arm]

//03 Automatic City

Automatic driving houses [Architectural design]

//04 The Smart Wall

Interactive space creation [Arduino, Emotion Recognition, Interaction]

Short-Term Projects //05 City Context

Classification of Chinese cities’ texture [Deep learning Self-organizing Maps]

//06 Manhattan Workers

Research of housing and transportation in Manhattan [Data Visualization]

Community CoGeneration Platform Team member: Jieyu Zhou Leshan Fu Instructor: Dr. Waishan Qiu

Participatory design is hands-on democracy in action. For over half a century it has guided us in understanding communities, honoring difference, creating vibrant neighborhoods and ecosystems, challenging environmental injustice, and fostering citizenship. However, design team have found it difficult for all the stakeholders to reach a consensus. The platform is intended to solve this problem by using deep learning for quick generating schemes and Grasshopper for rapid evaluation. A community in West Oakland is used as a case study.

Problems in Community Meeting

Difficult to incorporate different needs

The community in west Oakland has a site for urban renewal. However, residents, the government and developers have obstacles to reach a common ground in collective design process. Therefore, a platform for auto-generation is urged to quickly produce design schemes for the community.

Site Social Relationships in a meeting Residents Daily needs Social Media

Cond omin ium

Engagement

Developers We focus on market viability and business opportunities

ial

Big data

Soc

on t uti Sol nmen ver Go

Participatory design

eco no mi De cg vel oal op s ers

Simulation Education

Prototype for the new approach Data

Real-world Problems

Collecting (Smart sensing, crowdsourcing)

Awareness

Presenting (machine learning, augmented reality)

Understanding

Manipulation (Virtual reality, multiagent planning)

Residents

Intervention

Government Build green to support the Sustainable Oakland Program

We need to see this historic station from any angle.

Framework of Co-generation Platform Procedures of Generation

Interactive Module

Composition of the Installation

1 Leisure (Orange) 2 Amenity (Gray) 3 Shop (Blue)

4 Religion (Green) 5 Sports (Red) 6 Greenland (Pink)

Workflow

2. Model

3. Analysis

7 Landmark building in site 8 Site boundary 9 Number of the plan

10 Start and Clean button 11 Radar map 12 S o c i a l e c o n o m i c a l predicted data

13 Environmental analysis simulation

Components Projector

2. Assembling Blocks

1. Different programs of blocks

1. Discussion

The stakeholders can build up models with the pre-set blocks of different programs. The camera on the top will capture what they have built and then sent the image to the mainframe for next generation and evaluation.

Showing the site and analysis images

3. Generating

10

Webcam Capturing which blocks users put 11 9

Picture processing, generating and analyzing

7

6

12

8

5 4 3

4. Analysis

Mainframe

2

Evaluating scheme by environment, economy and ecology indicators

Users assemble blocks on the site model. Results are captured by camera

5. Redoing

Model

1 Not meet needs

Analysis

AI Generated Design Module

GAN ( Generative Adversarial Networks) Algorithm Pix2pix Algorithm In this step, 500 paired images (image A & image B) of appropriate census block groups’ plans(500m*500m) in American cities were collected by Python in Open Street Map. Image A shows the landuse and image B shows the building outline. These paired images are used to train the pix2pix model. Then input the user's sketch into the trained model to generate a plan.

Evaluation Module

Results Testing Catalogue

The evaluation system is created based on different stakeholders' needs, which can be divided to four parts, seven aspects.

Some of the evaluation is simulated in Grasshopper in Rhino, others are generated by dataset.

Residents

Job Offering

Government

Developers

View of the historic Station

Sunlight

Revenue

Eco-friendly

Economy

nt nm e o r i Env

Rooftop Solar Potential

Accessibility

CO2 Emission

Environment

Job Offering

Space

2

7 2.9

4

S p a ce

Eco-frie n

dly

7

3.1

1 4.8 2 Form: Data

1 Form: Data

Predicted by: dataset from Google Environmental Insights Explorer

Economy

Rating for Plan 19

3

9

Predicted by: dataset from U.S. Bureau of Labor Statistics and Data USA 3 Form: Model Simulation

Predicted by: Honeybee and Pedsim in Grasshopper

Evaluation Module Results Testing Catalogue

Sunlight

In the West Oakland project, there are 12 plans created by different stakeholders with the aid of the AI tool, pix2pix, and their evaluation is shown below. ① Plan ② Plan ③ Plan ④ Plan

View of Station

Accessiblity

①

②

③

④

1 4 8 11

Sunlight

Overall

View of Station

Accessiblity

Overall

1

7

7

2

8

8

3

9

9

4

10

10

5

11

11

6

12

12

110cm

Accessiblity Score: 7.1 100

12

90

11

10

7

6

5

4

3

2

1

Sunlight Score: 9.5 80

70

60

View of Station Score: 8.3

50

40

30

Paln number, Average score to chose the best one Highway to Sanfransico for commuters Different stakeholders

Ecology data and Economy data

Overall rating using radar map

16th Train Station started in 1912 Simulation Results in Grasshopper Honeybee and ladybug

20

Start Generation and clean bottom

Data Source: OpenStreetMap, U.S. Bureau of Labor Statistics and Data USA

10

1

1

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

160

170

Mechanical Rockery Team member: Jieyu Zhou

Instructor: Diego Pinochet Xu Zhang

People in the non-design field aided by robotic arms participate more easily in the model making process by the robot performing iteration and arranging complex spatial formation. Users simply place a rough structure and a robotic arm will complete it. Training by reinforcement learning in Unity 3D with a physical engine will ensure the stability of structure completed by the robot, and the layout of the components will just mimic the users’ outline as the robotic arm once be trained according to the datasets.

Mechanical Rockery

Understanding rockery building process Shape Grammar ange its color according to the kinematic velocity and shrink based on the object's location.

Rules

[ Rule 1 ]

Image Source: The Ming dynasty, painter Lin Youlin

[ Rule 3 ]

Computation

[ Rule 4 ]

[ Rule 5 ]

[ Rule 6 ]

ange its color according to the kinematic velocity and shrink based on the object's location.

Rules Applied

[

Results

Stone Types

Initial

1

Initial

3

1

5

2

1

1

3

1

1

Initial

2

4

6

4

2

Initial

3

4

3

4

2

+

Triditional process of making rockery

[ Rule 2 ]

]

1

[ ]

3

[

2

]

Rockery Selection

[

Cleaving Rocks Adding small ones Rotation and Addition

4

The rockery is an integral element of Chinese classical gardens.

To help the public to understand the space of rockery, the stones are abstracted as dodecahedrons to analyze different forms and combination styles with the method of "Shape Grammar."

]

1 2 3 4

① Zhuo Yu ② Dan Zhou ③ Tai Hu Stone ④ Cang Jian

Mechanical Rockery Building Rockery with Reinforcement Learning Trained by reinforcement learning in Unity 3D with a physical engine will ensure the stability of structure completed by the robot, and the layout of the components will just mimic the users’ outline as the robotic arm once be trained according to the datasets.

Reinforcement Learning State & Reward

Agent

Environment State Transition Action

Reinforcement learning let the agent take suitable action to maximize reward in a particular situation.

Multi-Agent Cooperation Academy External Communicator

Action State & Reward Agent

Action State & Reward Agent

Python Tensorflow

......

......

Action State & Reward Agent

Sharing Instantaneous Information eg. Reward/ Position/ Rotation/ Velocity/State

Cooperative agents share sensation, episodes and learning policies. This sharing speeds up learning at the cost of communication.

Initial Input: Cang Jian

Initial Input: Dan Zhou

Initial Input: Qiu Chuang

Output Training Model using ML

Output Training Model using ML

Output Training Model using ML

Offline Simulation with Reinforcement learnning

Mechanical Rockery

Force: 1

Force: 2

Force: 3

Force: 0.75

Force: 2

Force: 0.2

Force: 1

Force: 4 Force: 0.8

Force: 3

Force: 2

Force: 0.6

Force: 0.4

Results of Reinforcement Learning in Unity 3D ML-Agent and data on Tensorboard Learning Rate

Value Loss

Episode Length

Cumlative Reward

Mechanical Rockery

Simulation of the Robotic Arm in Unity3D

In Unity3D, a six-axis robotic arm is used to simulate the rockery building process, which can help manipulate the robotic arm in reality. The epistemic action with physical models encourage the unprofessional to explore the aura of space and engage in design more deeply.

Prototyping and Circuit

[ Kuka R900 ]

[ Gripper ] 3D-printed Grip objects with kuka

Robot Robot to grip objects

[ Kuka RC4 ]

[ Connection ] Connection Adapting piece for gripper

Controller Control Kuka and gripper

Kuka working radius

1

22'65

3

Building Process

4

2

1 Camera 2 Worktable

Interface Input 2

3 4

Kuka R900 Objects needed to load

Mechanical Rockery

Mechanical Rockery in Forbidden City

The mechanical rockery manifests a brand new way to understand the space forms of traditional rockery with reinforcement learning and the robotic arm.

Automatic City Team member: Siming Chen Jieyu Zhou Instructor: Pro. Mark Anderson

This design aims at creating ecosystems that are benchmarks how cities should be in future. Going vertically is a strategy to solve the problem of landuse shortage. There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing unit will have an elevator as its owner's car to transport them to everywhere in the city. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place. To push the boundaries of design, this project also used innovative habitat working models, materials, technology, close to zero land costs, a nomadic yet rooted lifestyle.

Automatic City

Self-driving housing units as transportation

This design aims at creating ecosystems that are benchmarks how cities should be in future. With a faster transit and a more connected world the need to stay rooted at one location will go away and futures will be more transit/ mobile.Faster transportation techniques and connectivity + collaboration, has made it possible to look beyond boundaries of cities Live

Plants

Water

Resistance

City Growing Strategy

Expension

Sahara

2.5 mile

Skyscraper

This design aims at creating ecosystems that are benchmarks how cities should be in future one location will go away and futures will be more transite in future one location will go away and

Prototype Habitat

Terrain

Energy

Sandstorm

Rainfall

Transit

For 1,000 people 0.5*0.5mi

Road

Layer for Housing Units 1.Self Driving Home

2.Problems

12 million

Americans are blind or have low vision

3.Computer Vision

4.Construction Tech

Highway

79%

of seniors in car dependent communities

0 100 m

54 hours

wasted in traffic year per person

2.5 mile

500 m

1 km

Automatic City

Self-driving housing units as transportation

Facade

A u t o m a t i c Fa ç a d e , c o o l i n g system and building strategy applying bacteria are introduced to this project.

The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place.

Elevators

Grow human civilization in synchronus with nature + technology + planet

Diagrams of Units

Sahara Skyscrapers

Housing or working Modules

Transportation Module

This design exercise can be considered similar to colonizing a new earth with technology of today.

Linear combination modes

Working sapce

The structure and module of working space will remain the same with the housing units

Ring combination modes

Linear combination modes

Diagrams of Single Skyscrapers

Transit

Form 1

Form 3

Form 2

going vertically is a strategy to solve the problem of land-use shortage.

Automatic-driven elevator.

Diagrams of Skyscrapers and Joints

Housing unit

There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing units

Transportation Module

Single Building

Connection

Greenspace

It is a space in all skyscrapers that community members can enjoy their time

Automatic City

Self-driving housing units as transportation

There will not be any cars or roads in the city. Instead, with the technology of automatic driving, each housing unit will have an elevator as its owner's car to transport them to everywhere in the city. The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place.

Module units

Housing Units

Elevator

Working and playing

Cooling System

Automatic Façade

Building strategy

Plans for Working modules and housing units

0

Layer for Housing Units

Layer for Working

1

5

10 m

Automatic City

Programs

Self-driving housing units as transportation

Center

Center

The structure and module of working space will remain the same with those of housing units. Elevator shafts make it convinient for "cars" to transport people from their houses to the working place.

Work Housing Leisure Transit

Transit Housing Work Greenland Leisure

Rendering for the transit

Transit

Transit

The Smart Wall Team member: Jieyu Zhou Baoliang Yu Instructor: Pro. Li Li

Making authentic models and creating space on their own offers a chance for people in non-design field to experience the aura, aesthetic, and mystery of architecture. In other words, it is a heuristic way to understand space, form and style and participate deeply in design process. In my project, the smart wall can change its color according to users’ facial expressions and twist based on users’ body movement. In this way, people less-informed of design knowledge engage in design process more actively and have the opportunity to experience the aura of space through lighting and motion of this wall.

The Smart Wall

How do we perceive space?

In the common sense of people, walls are always still with no emotion. But can we bring a new definition to the wall? That is where the idea of our project began. We want to make the wall talk to people through waving and shining.

1 Design Concept Diagram

2 The Evolution of Space

① ② ③

Classical Modernism Cyberspace

①

②

Different Perception of Space

How do emotion influence we perceive space?

Emotional Recognition

The same space but different results

It is a just normal place for me.

Collective memorries and difference

This classroom recalls me the time I spent with friends. I feel nostalgic for the old days.

I feel depressed today.

All I see is blue.

A space that can act according to your emotion Different perception system

③

The Smart Wall

Workflow Diagram

Algorithms and Framework

[ Motor End-Effector ]

[ Lighting End-Effector ] 8

9

... ...

6

7

... ...

Prototyping and Circuit

[ LED Strip ]

[ Fiber Optics ]

[ Arduino ]

[ Raspberry Pi ]

Motion Controll motion with grids

Lighting Show colors with fiber optics

Lighting and Motion Colors and enlarge motion

Controller Controll motors and LEDs

Controller Emotion Recognization

8

5 +

[ Servo Motor ]

... ... +

... ...

189

+ -

... ...

[ Digital Control ]

6

4 Power [220V]

+ -

Central Controller 4.Arduino Mega 2560

4

Motion 5.Adafruit 16-Channel 12Bit PWM Servo Driver 6.Mg996 Servo Motor 7.Flexible Gribs

5 2 1

Arduino IDE Fast LED Library Adafruit PWM Servo Driver Library

Lighting 8.WS2812B LED Strips 9.Fiber Optics

3

Serial Communication

[ Raspberry Pi ] 1

2

1 Power 2 Mg996 Servo Motor 3 Adafruit PWM Servo Driver

4 WS2812B LED Strips 5 Arduino Mega 2560 6 Raspberry Pi 4

Input 1.Fan 2.Camera Module Central Controller

3

Emotion Recognition [OpenCV,Dlib] Object Tracking [Optical Flow] Linux

Power [220V]

+ -

The Smart Wall How do we perceive space?

Optical flow is the pattern of apparent motion of objects, in a visual scene caused by the relative motion. It is used to detect motion in this project. With the use of OpenCV and Dlib, 68 points we re dete cte d as fa c i a l landmark annotation. Using these points, facial features were analyzed to recognize different emotions.These points, facial features were analyzed different emotions and sent them to Arduino.

The smart wall can change its color according to users’ facial expressions and twist based on users’ body movement. In this way, people less-informed of design knowledge engage in design process more actively and have the opportunity to experience the aura of space through lighting and motion of this wall.

Motion Mode: Optical Flow

Emotion Mode: Emotion Recognition 1

Annotate 68 facial landmarks with the Dlib trained model

Open the camera, Compress image, and grayscale

2

OpenCV Dlib, facial landmark annotation

3

Open the camera, Compress image, and grayscale

4

Annotate 68 facial landmarks with the Dlib trained model

5

Calculate degree of raised eyebrows and frowning, eye opening distance and mouth opening distance

5

Emotion Recognition

6

Recognize the emotion, and transmit the code to Arduino

7

Skin 1. Fiber Optics 2. 3D Printed Grids Structure 3. Wooden Planks 4. Frame with motors 5. Central Controller 6.Arduino Mega 2560 7.Raspberry Pi 4

1

2

3

4

The Smart Wall

① Motion Mode

How do we perceive space?

If there is any moving object detected, the wall will change its color according to the kinematic velocity and shrink based on the object's location.

Camera

3

speed ++++ Very Fast The color will be brighter and warmer.

2

speed ++ Fast The color will be less birght and warm.

1 speed + Slow The color will be darker and cooler.

Motion Mode Change color and shrink by the velocity

Chronological Sequence

No moving object, showed preset color Palette

Moving object tracked

More Active

The Smart Wall

② Emotion Mode If there is no moving object but faces are detected, the wall will change its color according to emtion.

How do we perceive space?

Happy

Natural

Facial expression

Facial expression

LED pattern: Purple

LED pattern: Yellow

Natural

Happy

[Facial expression: Happy]

[Facial expression: Natural]

Angry

Amazing

Facial expression

Facial expression

LED pattern: Pink

LED pattern: Blue

Amazing

Angry

[Facial expression: Angry]

[Facial expression: Amazing]

Short-Term Projects City Context Team member: Baoliang Yu Jieyu Zhou Instructor: Pro. Li Li

In the study of commercial environment, I used one of the machine learning methods, Self-organizing Maps to classify all shopping malls’ texture in China, aimed at interpreting sophisticated city contexts to the grassroots and encouraging them to engage in the city regeneration process.

Short-Term Projects Manhattan Workers Team member: Tong Ji Jieyu Zhou Instructor: Pro. Karen Chapple

This research focuses on housing and transportation data in New York, making it easier for the public to understand the complex housing market and transportation situation.

With computational analysis tools, Matplotlib, and geographic visualization tools, Carto, complex raw data turned into interpretable, actionable information in the process of participatory design.