Spaces of Reciprocity

spaces of reciprocity

Research + Design portfolio Jade Tang ADS6 RCA 2019-20 1

Film still, Cliff

2

abstract ~ The project explores reciprocal relationships between the corporeal and the digital, and how these can begin to trigger alternative understandings of producing and perceiving space. Jorn Utzon’s seminal project “Can Lis”, is recorded via a series of experiments through bodily and technological means , documented and explored through film. Moments of reciprocity between these two perspectives are materialised and performed through the design of three scenarios, in order to imagine how an ‘other’ architecture may emerge through these alternative encounters between body and technology. In doing so, they challenge the Western body-environment dualism and instead consider Eastern principles based on an interconnectedness between human and nature, particularly questioning concepts such as form, circulation and ambience.

These alternative scenarios allow us to speculate how the reconsideration of ocular experience, aural perception and emotion has the potential to reshape our relationship to the environment and the human body, using architecture as a fundamental tool.

3

4

With s p ecia l tha nk s to: My tutor s Clara Kraft Is ono, Guan Lee, Sato s h i Is o n o Gr y ms dy k e Far m Lea D’albronn Yin Man Tang Adam Holloway Alek s andra Sobczy k Chris topher Fis chlein Marie Rams ing Sunjna Mullick

5

6

C o n ten ts ~ I n tro d u c ti o n 0 0 Pro j ect stru ctu re + s u mma ries

p. 8

01 Two bo di es p. 1 5 02 Reci pro c ity p. 2 1

T h e 3 G estu res 03 D ev i ces p. 3 3 04 L o c ati o n p. 4 5

05

06

[Gesture 1] p.53

Sh ì j ué/Sig ht i) The readings ii) The scenario

[Gesture 2] p.89

T ī n g j ué/ Sou nd i) The readings ii) The scenario

0 7 [Gesture 3] p.121 Q i /A i r

i) The readings ii) The scenario

~

08 A ppen di x p. 1 5 5

i) Previous design development ii) Other research

09 Bi bli o g r ap hy p. 1 6 3 7

Summ

INTROD

~ ~

8

mary

UCTION

~ ~

9

0

2 bo 0

Re c i p 0

D ev 0

Loca

0

05 [ G e s t u re 1 ]

[Gest

Tīngjué/

Shìjué/Sight

Sou

Sight

C i rc u

Fo r m

SITE: C ave, S a n t a ny i

SIT L i v i n g ro o

Technology: Structured light 3D scanning Body: Eyes, Balance, movement Actions: Scanning and walking

3D Scanning

10

Performative fabrication

Movement

Technology: Binaru Body: Hands and Actions: Sound reco

Binaural recording

Immer sound

00: Project structure + summaries

01

odies 2

p ro c i t y 3

vices 4

at i o n

06

07

t u re 2 ]

[Gesture 3]

/ Sound

qi/air

und

Emotion

u l at i o n

Ambience

SITE: Observatory, Can Lis

TE: om, Can Lis

ual sound recording Ears ording and touching

sive 3D dscape

Knocking

Technology: Pulse sensor Body: Lungs and Heart Actions: Breathing and walking

Pulse readings

Film construction

Breath

11

Summary of

[ G e s t u re 1 ]

Shìjué/Sight GESTURE 1 is situated in the cave beneath Can Lis. This gesture explores the reciprocities between two ways of seeing: digital scanning technology and ocular experience.

These two perspectives, are superimposed, materialised and performed through a process of fabrication. The robot arm follows the original topography of the cave, and is interrupted by live movements of my hand at specific time intervals, which are based on my bodily experience of the cave.

This process considers the potential of performative fabrication as a real-time design method that consists of an interrelated network between technology, body and place, challenging Western body-environment dualism in the generation of form.

12

[Gest

Tīngjué/

GESTURE 2, is situated in the Living room between binaural recording technology and circulation. These two ways of listening a design of an immersive 3d soundscape exp walking. During the experience, performers , as the digital soundscape is superimposed moments allow each participant to define an entirely unique response that is depend pla

This overlapping of the situated and the between the stage and the screen, proposed of sonic construction can be used as a real coexistence between site, technology and th sonic conditio

00 : Project structure + summaries

f 3 g e s t u re s

t u re 2 ]

/ Sound

m of Can Lis and explores the reciprocities d live aural experience, questioning ideas of are materialised and performed through the perience, asking viewers to respond through s encounter various moments of interference d onto their immediate environment. These their own path through the space, creating ent on time, interpretation, materiality and ace.

constructed, breaks down the distinction d and realised. It questions how this method l-time collective design tool that considers a he body, reconnecting us to the rhythms and on of the earth.

[ G e s t u re 3 ]

qi/air GESTURE 3 is situated in the observatory and explores the reciprocities between two expressions of emotion: the pulse and breath, both of which are invisible responses to the surrounding enviroment. These two forms of expression, are documented and materialised through an installation and film, exploring the potential of using emotion as a material to explore ambience.

The installation creates a scenario where the overall ambience/result is defined by these two forms of expression, demonstrating a process where body, technology and place work collaboratively to generate an image. The speed and movement of salt particles are defined by the live breath rate of the body during the filming process, while the rhythm/frame rate of the projected light is based on the recorded pulse rate experienced in Can Lis, therefore defining when the particles are illuminated and visible to the camera.

13

14

01

Two bodies

15

Two b o d i es: two b ody -env ironment relationships

I began looking at the perception of the human body and discovered that there was a particular divergence between Western and Eastern culture. Shigehisa Kuriyama wrote about a particular divergence of Greek and Chinese medicine, outlining how these two cultures differ in their styles of sight, touch, and being. These diverging attitudes to the body also represented their relationships to the environment. The West tend to perceive the body as distinct from the environment, widening the gap between the body and the environment.

However the Chinese, believe that the body is simply a part of a wider network of relations in the universe, illustrating an interconnectedness of all entities.

16

01: 2 bodies

(Fig.1) - Vesalius’s Fabrica (1543)

(Fig.2) - Shisijing fahui (1341)

17

Sight: measurement, form , appearance Both cultures differed in their methods of measuring and perceiving the form of the body, which had led to a divergence in how the body should be seen and imagined.

The Greeks had a large interest in muscles, and saw them as distinct from flesh and tendons. Students who were studying art were required to understand the anatomy of the body and remain aware of the presence of bones and muscles in order to draw and represent it. 1 This made it hard to distinguish between what was imagined through previous knowledge of internal anatomy and what was seen in the real model that was in front of them. In a textbook on drawing, by Charles Antoine Jombert, he states that a beginner sees almost no muscles in a nude body and envisionary musculature is an acquired skill. 2 This requirement to imagine the inner anatomy of the body with basic knowledge of the muscles was also related to the Greek desire to dissect the body to understand it – highlighting the prominence in anatomical seeing in the culture. This demonstrated a particular way of looking at and into the body, which was the birth of a certain visual style. 3 For the Chinese on the other hand, dissection did not have a large presence in Medicine, and continued without a desire for it for thousands of years. 4 Chinese doctors gazed past the observations made by the Greek method of dissection and instead imagined invisible features that could not be established / acknowledged through dissection.

This engaged a gaze that trained on living persons rather than on lifeless corpses, where they found importance in scrutinising the living rather than the dead.

Although the Chinese did take part in the dissection of corpses, it never gained dominance in understanding the body. The dissection of Wangsun Qing was the first and potentially the only dissection ever conducted in ancient China. 5 The dissectors were only concerned with measuring and weighing the internal organs, tracing the course of blood vessels, where the anatomy was approached like a cosmic enquiry. 6 Details were carefully studied, not focused on the form or structure, but instead they were interested in taking measurements of the body and its parts: ‘the number of a persons limbs matched with the four seasons and the four directions, the five zang with the five planets, the twelve conduits streaming through the body with the twelve rivers bearing life to the land’. 7 This rare example demonstrates that there is more than one way to study the physical form of the body: the Chinese did not see the nerves and muscles that the Greeks saw and found interesting, but focused on measurements instead.

A matter of governance across the physical body parts was also a crucial factor in understanding the divergence in the understanding of anatomy across both cultures. The palpation of the ‘mo’, similar to circulation, in Chinese culture gave insight to all the organs equally not just mainly the heart. Unlike nerves and blood vessels, the Chinese ‘mo’ formed a circle with no controlling source, where it began from and returned to simply a place (cun kou). 8 Whereas, the Greeks focused on the discovery or the main ruler/controller, which was often only ever shifting between the brain and the heart. 9 Greek dissectors assumed that this was what was understanding the body’s structure was about. This did not exist in Chinese thinking however, it was a matter of power circulation. The Chinese conception of the body differed not just by the multiplicity and quality of governance but most importantly an alternative conception of governance.

Where the heart controlled all the other arteries in the body, and the brain controlled the muscles in the Greek body, the Chinese percevied a different organisation of internal parts, and were generally more concerned with an indirectness. 10

1 Ibid.

2 Ibid.

3 Ibid.

4 Ibid.

5 Shigehisa Kuriyama, The Expressiveness of the Body: And the Divergence of Greek and Chinese Medicine (New York: Zone Books, 2002) 6 Ibid.

7 Ibid.

8 Ibid.

9 Ibid.

10 Shigehisa Kuriyama, The Expressiveness of the Body: And the Divergence of Greek and Chinese Medicine (New York: Zone Books, 2002)

18

It seems that Greek anatomical seeing involved a specific urge/desire to understand the body through dissection. Although the Chinese also had a similar procedure, the divergence of the two cultures seem to have been caused by different objectives. It was the matter of examining the past and future, against the present state of the body, where the Chinese were more concerned with invisible shifts in power amongst body parts and not driven by the visual form and immediate functions that were expressed by the body.

Touch: flow, time and rhythm Both cultures differed in their styles of touching as a method of diagnosis, and how issues sensed through touch were to be interpreted. The process of palpation, the process of using ones hands to check the body, was used to diagnose changes of flow, rhythm and time within the body. The Chinese used palpation to examine ‘mo’ which resembled circulation, unlike the West who examined the ‘pulse’. 1 They analysed the ‘mo’ by tracking changes in how it felt, where the doctors focused on 6 changes, whether mo were full (ying) or empty (xu) quiet (jing) or moving (dong) slippery (hua) rough (se). 2 They were also interested in studying the palpation of qi, and was regularly compared to mo, where mo did not relate to any particular artery or vein and was decided by the inner experience of pain. 3 On the other hand, around 129-200 c.e , the Greeks believed there was one pulse and consisted of 4 parts: diastole, preceding systole, the style and the rest. 4 They visualised a tublar artery rising and falling in order to understand how the pulse ‘felt’. This demonstrates how the two cultures used the same gesture of touch however interpreted them differently. It seemed that was the Chinese focused on the inner experience of the body to understand concepts of flow and rhythm of ‘Mo’, whereas the Greeks visualised the process of the artery in order to understand how the pulse ‘felt’.

Being: environment, ambience , air Both blood and breath (or qi) were seen as crucial elements that sustained life in both cultures. However, the differing attitudes towards these elements caused a divergence, despite having similar methodologies in dealing with them.

Up until the 19th century, the letting of blood was one of the most common ways of caring for the body in the West and concentrated on the release of excess to restore the body to a healthy state. 5 Similarly, Chinese healers punctured abscesses and let blood with bladed stone or bronze scalpals, bianshi before the introduction of acupuncture. They believed that sickness was related to or began with the principle of ‘emptiness’, xu, and was in parallel to ‘fullness’ shi, where ‘xushi’ represented a fullness in health. 6 This principle framed the basis of acupuncture, where an empty body would be open to invasion and invited shi: “shi is when qi comes in, xu is when qi goes out”, and these terms of fullness and emptiness helped to define or understand the imbalances in internal power. 7 These imbalances would relate to those of the universe, as they saw the body was only one element in a universe of interrelated entities. 8 qi is thought to flow through earth. ‘ from wind’ or environmental influence, flowing to the various parts of the body through pathways known as meridians, which do not have objective existence according to bio medicine. 9 Using acupuncture, hundreads of points in the body (tsubo) can be stimulated, in order to affect substances flowing through meridians. 10 In this case, both cultures, saw an imbalance of blood or breath (qi) as a sign of disruption in the overall health of the body however chose to interpret the signs of imbalance in different ways. However, this also suggested their diverging perceptions of the relationship of the body with the surrounding environment.

Where the Greeks were looking for factors that were rooted from the body and not beyond, whereas the Chinese related it as part of a wider network of interrelated points of the universe / functioning of the body was explained in relation to the physical world. 11 1 Ibid.

2 Ibid.

3 Ibid.

4 Ibid.

5 ‘The History of Bloodletting’ <https://www.bcmj.org/premise/history-bloodletting> [accessed 10 April 2020] 6 Shigehisa Kuriyama, The Expressiveness of the Body: And the Divergence of Greek and Chinese Medicine (New York: Zone Books, 2002) 7 Michel Feher, Fragments for a History of the Human Body, 2 (New York: Cambridge, Mass. distributed by MIT Press, 1989); Kisho Kurokawa, Intercultural Architecture : The Philosophy of Symbiosis (London: Academy Editions, 1991); 8 Ibid.

9 Ibid.

10 Ibid.

11 Shigehisa Kuriyama, The Expressiveness of the Body: And the Divergence of Greek and Chinese Medicine (New York: Zone Books, 2002)

01: 2 bodies

(Fig.3) - Pulse taking in ancient China

(Fig.4) - Medieval Pulse taking

19

20

02

Reciprocity /ˌrɛsɪˈprɒsɪti/ [ the practice of exchanging things with others for mutual benefit, especially privileges granted by one country or organization to another.]

21

Space of reciprocity: Body casts Model, Clay 20 x 8 cm In attempt to capture the space of reciprocity, a series of slip casts were formed of my arm. An imprint of my forearm was made in soil of varying moisture levels, forming a void for the slip to be poured into. After 30 minutes, the excess slip was removed to reveal a thin skin along the walls of void.

22

23

Body casts 5 units

24

Slip casting process Soil , slip

25

Ab trainer

The act of reciprocity: Gesture Both of these objects trigger a specfic gesture on interaction with the human body. Without the presence of the object or the human body, the gesture or action does not exist.

26

02: Reciprocity

Movement of body on interaction

Primitive and contemporary hammer

27

Measuring reciprocity The process of baking sourdough was documented in two ways: one as gestures of the human body, ‘human nature’, and one documented based on the standard metric system, ‘measured nature’ .

What is this third space between? How can we capture /measure it?

28

Body Gestures

Xbox kinect scanner, Digital coordinates / points

Ruler, Metric unit system

â&#x20AC;&#x2DC;Human nature vs Measured natureâ&#x20AC;&#x2122; Scanning, drawing Duration: 2 minutes

29

THE EXPERIMEN

~

Based on Kuriyamaâ&#x20AC;&#x2122;s discuss on 3 areas for my project, usi as mediums to cond

30

NTS: 3 GESTURES ~

sions, I decided to concentrate ing sight , sound and emotion duct my experiments.

31

32

03

Devices A series of custom made devices and technology were used to take a series of readings of my experience.

33

The recorder

3

7

The controller

12

1

6

8

4

5

15 11

Role:

Role:

1. To document experience through technological and bodily means.

1. To assist the recorder in documenting experience by controlling technological devices.

2. To guide and support the recorder during the recording process.

The characters Each gesture needs to be performed by two actors : one actor takes the corporeal reading, the other is a receiver who controls the technology. They are both codependent, as one cannot perform without the other present.

34

03: Devices

add pulse reader

Re c i p ro c a l S t at e s

[1] Title

15 5 4

2

3

1 6

14

9

8 7 13 12

11

1. listening cone (R) 2. listening cone (L) 3. eye goggles 4. binaural microphone 5. binaural microphone 6. Recorder 7. binaural microphone 8. binaural microphone 9. lidar scanner 10. lidar stand

10

receiver (R) receiver (L)

11.

12.

13.

14.

15.

ipad headphones suit 1 suit 2 arduino board + pulse sensor

transmitter (L) transmitter (R)

x

z

35

A

Transparent PVC pockets to hold the sound recording equipment, so the screen can be easily seen or accessed.

Arm straps help to guide the wires for the microphones to be worn.

160 - 170 mm

The colour of the material was chosen in response to the house and the surrounding landscape , where the body, landscape and architcture appear in harmony with one another.

The suits The suits were designed and made to accomodate the binarual technology carried by both characters. These were made in collaboration with my grandma.

36

03: Devices

2 garments

Process sketch

Fabric, PVC

37

Interior of cone wrapped in mylar to help reflect sound.

150mm

600mm

Adjustable wire headband to allow the cones to be worn.

Extra support around the ear area to minimise noise loss.

The ears The cones were designed to be worn by the recorder, in order to amplify the sound heard by each ear.

38

03: Devices

Object Mountboard, wire headband

39

Reference: Rebecca Horn From a personal experience of being hospitalised, these bodily extensions emerged from her struggle to express herself and explore her surroundings.

These devices were designed for her awakening as she returned back to the real world.

40

03: Devices

41

Elastic band

Flesh coloured tape for extra cushioning

Chemiwood

The goggles These goggles were digitally modelled and cut with a cnc machine.

42

03: Devices

Object Elastic, chemiwood, tape

43

44

04

Location Can Lis Mallorca, Spain

45

Screenshot from Goog l e ea r th 2020

46

04: Location

Can Lis, Jorn Utzon Mallorca, Spain ~ Throughout his life Jorn Utzon resisted a Eurocentric understanding of architecture and turned to a variety of cultures, particularly eastern readings to find an ‘other’ understanding of how we build and experience space. He rejected a reductive reading of space as set out by modernist dogma, to build a transcultural ground from which to build from. His seminal project “Can Lis” his own house Mallorca, Spain, was the site where he played out many of these ideas and became the location for my readings.

47

Plan Digital hand drawing

West Elevation Digital hand drawing 48

04: Location

Sight: Rhythm of the sun Utzon manipulated the form of the house to achieve the gradual changes of light throughout the day, avoiding distinct contrasts between light and shade. This ability to constantly trace the whole sun path from the interior of the house, allows for the reestablishment of the bond between human being and place, using the house enclosure to facilitate a constant awareness of the sun. These similar interactions with light seem to have been influenced by his experiences in the cave below the house.

Touch: Earth work The materiality of the house suggests that the walls and ground are one element, as if an extension to the earth. The natural qualities of the sandstone and its relationship to sound mimics primordial sensations of the cave, suggesting a certain sense of rhythm throughout the house.

Ambience: Interconnectedness Utzons desire to form a harmony between building and landscape was also achieved by triggering emotional sensations of fear and safety. By locating the house on the edge of the cliff, the house was exposed to the sea, sky and land where Utzon used this exposure to the changing atmospheric landscape/phenomena to create moments that shifted between enclosure and exposure, windy or stillness, hot and cold. This fluctuation in emotion emphasises a closeness with the landscape through a reminder of the constant transformation of the environment.

49

2 3

0 Introduction

1 Gesture 1: Cave

Cinematic landscape: Body and technology The film progresses through the landscape via the associated sites for each gesture: the cave, living room and observatory.

A hybrid landscape was generated using a combination of filmed footage and animation , where technology was used to generate scenes that were not accessible / captured by the body and camera.

50

0

1

2 Gesture 2: Living room

3 Gesture 3: Observatory

C a m er a p ath 51

52

05 Gesture 1

ShijiuĂŠ/Sight i) The readings ii) The scenario

53

54

71,890,950 points 34 steps

55

05 [ G e s t u re 1 ]

ShijiuĂŠ/Sight Fo r m

SITE: C ave, S a n t a ny i Technology: Structured light 3D scanning Body: Eyes, Balance, movement Actions: Scanning and walking

3D Scanning

56

Performative fabrication

Movement

Summary

SUMMARY ~ ~ GESTURE 1 is situated in the cave beneath Can Lis. This gesture explores the reciprocities between two ways of seeing: digital scanning technology and ocular experience.

These two perspectives, are superimposed, materialised and performed through a process of fabrication. The robot arm follows the original topography of the cave, and is interrupted by live movements of my hand at specific time intervals, which are based on my bodily experience of the cave.

This process considers the potential of performative fabrication as a real-time design method that consists of an interrelated network between technology, body and place, challenging Western body-environment dualism in the generation of form.

57

00

Introduction

Transition to Gesture 1 58

1 Gesture 1: Cave

0

1

2 Gesture 2: Living room

3 Gesture 3: Observatory

C a m er a p ath 59

Film stills : Transition to Gesture 1 60

05: Gesture 1

61

62

i) THE READINGS ~

Gesture 1 is situated in the cave beneath Can Lis. The first reading utilises 3D scanning technology to create a point cloud of the surface environment. The second reading interprets the ocular experience of a person as they walk through the space documented as a drawing, in order to explore an alternative understanding of form.

63

Walking Photograph T=10 min Cave

Body: Walking The position and movement of the body in space is determined by agility of the eyes to establish distances between surfaces to generate a mental understanding of the space. Each position of the body relates to a unique space-time relationship that is constantly shifting and unpredictable.

64

05: Gesture 1

Film still Cave

Technology: Lidar scanning For the scanner, the form of the point cloud is dictated by the speed of the sensor to record reflected light beams and coordinate registration, where each point has a unique fixed space-time relationship, that is based on a set unit system set by the technology. Â

65

Film stills

Body: Walking 66

05: Gesture 1

Film stills of point cloud

Technology: Lidar scanning 67

L

R

L L R L R

L R L R

L

R

R R

R

RL R L L R

L R

L

L R

L

L

R

R

L

L

R

L

d

L

R R

R u

L

L R

T=0 min

Plan view

T = 0 min

T = 10 min L

R

R

L

L

R

L

R

L

R

R

R

L

L

R

R

L

L

R

L

R

L

R

R

L

L

L

R

R

L

R

L

R

L

R

R

L

L Balance

Body: Walking The movements of the walk are documented as a change in balance experienced by the body.

68

05: Gesture 1

‘Look around.

Its windy and the ground is slippery.

I can feel the mist of the sea on my skin.

We are walking slowly, staying close to each other and close to the ground .’

Transcript of experience

69

Plan view

T = 0 min

T = 10 min 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360

Degrees

Technology: Lidar scanning The scanner completes a full 360 degrees turn for each scan, where the time of the scan is determined by the reflected light beams. Each reflected light beam is registered as an individual point in space.

70

X

Y

Z

Raw point cloud coordinates

71

Technology: Lidar scanning The processing of the raw point cloud data involves a series of steps, in order to translate the data into a desired format. To generate the drawings above, a python script was used to place a sphere at the location of each point, in order to be translated into a two-dimensional drawing.

72

05: Gesture 1

73

74

ii) THE SCENARIO

75

Film stills Duration: 45 seconds 76

77

The scenario

These two ways of seeing, 3D scanning and ocular experience, are superimposed, materialised and performed through a process of fabrication.

The robot arm follows the original topography of the cave, and is interrupted by live movements of my hand at specific time intervals, which are based on my bodily experience of the cave.

This process generates a superimposed set of movements/tool paths, and considers the potential of performative fabrication as a real-time design method that consists of an interrelated network between technology, body and place.

78

05: Gesture 1

1 8

4

3

2

4

6 5

12

13

10 7

9

7

11

1.

2.

3.

4.

5.

6.

7.

Small robot arm Robot arm controller Laptop Torch Model of cave Actor Camera assistant

8. Fabric Backdrop 9. Camera position 1: Wide 10. Camera position 2: Close up 11. Lighting 1 12. Lighting 2 13. Canon 5D Camera 79

Motion to robot arm In order for the robot arm to interpret and understand my movements, a motion tracking system was generated using a grasshopper script that translated my movements into a format that could be interpreted by the technology.

80

05: Gesture 1

Motion tracking process

Grasshopper script

Assisted and advised by Adam Holloway

81

Cave fragment: Model 550 x 340 x 190mm CNC , polystyrene, paint A fragment was extracted from the orignal cave scan to generate this model used in the film.

82

05: Gesture 1

83

Superimposed movements The combination of hand movements and the original topography of the cave generated a new set of toolpaths. The irregulaties in the geometry demonstrate moments where these two perspectives intersect.

84

05: Gesture 1

A superimposed tool path

85

The set Grymsdyke Farm

86

05: Gesture 1

87

88

06 Gesture 2

Tīngjué/Sound i) The readings ii) The scenario

89

90

46 recordings 27 knocks

91

06 [ G e s t u re 2 ]

TÄŤngjuĂŠ/ Sound C i rc u l at i o n

SITE: L i v i n g ro o m , C a n L i s Technology: Binarual sound recording Body: Hands and Ears Actions: Sound recording and knocking

Binaural recording

92

Immersive 3D soundscape

Knocking

Summary

SUMMARY ~ ~

GESTURE 2, is situated in the Living room of Can Lis and explores the reciprocities between binaural recording technology and live aural experience, questioning ideas of circulation. These two ways of listening are materialised and performed through the design of an immersive 3d soundscape experience, where viewers are invited to respond through walking. During the experience, they encounter various moments of interference as the digital soundscape is superimposed onto their immediate environment. These moments allow each participant to define their own path through the space, creating an entirely unique response that is dependent on time, interpretation, materiality and place.

This overlapping of the situated and the constructed, breaks down the distinction between the stage and the screen, proposed and realised. It questions how this method of sonic construction can be used as a real-time collective design tool that considers a coexistence between site, technology and the body, reconnecting us to the rhythms and sonic condition of the earth.

93

2

0 Introduction

Transition to Gesture 2 94

1 Gesture 1: Cave

1

2 Gesture 2: Living room

3 Gesture 3: Observatory

C a m er a p ath 95

Film stills : Transition to Gesture 2 96

97

98

i) THE READINGS ~

Gesture 2 is situated in the living room in Can lis. The first reading is taken using Binaural recording technology that records the aural experience heard by the left and right ear respectively. The second reading documents the aural experience heard while knocking on surfaces in the space into a two dimensional soundscape drawing.

99

Knocking Film still T=2min

Body: Knocking For the body, the sound created by the knocking corresponds to the specific qualities of the surface material that the hand comes into contact with. The body is then provided with these aural cues that allows the body to respond and behave depending on individual perception. This experience is entirely specific to the time and position of the body.

100

06: Gesture 2

Walking Film still T=2min

Technology: Binaural recording Binaural recording technology records the sound heard by each ear as two separate channels. A microphone is attached directly onto or close to each ear, where subtle changes such as head and hair movement can even be captured.

101

A

B

Plan of Living room

Body: Knocking We interpret our surroundings based on Interaural time difference ( delay between ears ), interaural intensity difference caused by head shadowing effect, and reflections. The hand drawing interprets perceived changes in distance between my body and the surrounding walls as I moved through the space.

102

10th December 2019 , 15:30 Can Lis Route A

Time

103

10th December 2019 , 15:30 Can Lis

Left

Right

Technology: Binaural recording This process of recording allows us to separate the aural experience into 2 sepearate channels, where normally our natural aural experience is perceived as a singular source.

The data was input into a sound editing software and exported as raw sound level data, where subtle changes are made more apparent.

104

RIGHT EAR

LEFT EAR sample no.

db

sample no.

db

Sound levels

105

Assisted and advised by Roddy Canas

Research and testing : Binarual recording device Before the visit to Can Lis, a series of tests were conducted to generate a better understanding of the binaural recording technology using a preassembled device.

106

06: Gesture 2

Assistant with sound source

2m

1m

Recorder with binarual device

Test location

Testing process diagram ~ Space: 5m x 5m x 3m Moving Image Studios Total 16 tests

107

108

ii) THE SCENARIO

109

Film stills Duration: 1:45 minutes 110

111

112

113

The scenario These two perspectives are materialised and performed through the design of an immersive 3d soundscape experience, taking viewers on a walk that explores the potential of using these two ways of listening as a design tool.

Viewers are invited to respond to the soundscape through walking. During the experience, they encounter various moments of interference as the digital soundscape is superimposed onto their immediate environment. These moments allow each participant to define their own path through the space, creating an entirely unique response that is dependent on time, interpretation, materiality and place.

This overlapping of the situated and the constructed, breaks down the distinction between the stage and the screen, proposed and realised. It questions how this method of sonic construction can be used as a real-time collective design tool that considers a coexistence between site, technology and the body.

114

06: Gesture 2

2

3

1

4

Concept plan 1.

2.

3.

4.

Viewer Viewing source Headphones Performance route

115

The soundscape 116

06: Gesture 2

Screenshot: Editing a sound recording in Adobe Audition, that experiments with playing two different sounds in each ear.

The viewer is immersed within a composed soundscape that is a reconstruction of the aural experience using a combination of binaural and ambisonic recordings, aswell as other recordings that mimic the materiality of the site. The materiality of the soundscape attempts to mimic similar sensations of primordial sensations of Can Lis, reconnecting us to the materiality, rhythms and sonic condition of the earth.

117

118

Process 119

120

07 Gesture 3

qi/air i) The readings ii) The scenario

121

122

氣 / ai r The Chinese word ‘qi’ (氣) has been one of the main principles that the Chinese use to describe and think about the world, they believe that all life is constituted of qi, and continues to dominate principles of the land, body and city today. Qi is the material force or psychophysical process and transformation that connects all life systems, an interconnectedness of all entities.

A literal translation to English language would be ‘air’ or ‘breath’, however also refers to ideas such as ‘flow’, ‘energy’ and ‘life force’ in Chinese culture that is used in various contexts. From a personal upbringing, the word was most commonly used to describe the concept of ‘hot air’ and ‘cold air’ , that referred to healthy or unhealthy state of the body. However, these terms are not understood or used in the same way in English language. While air is normally used as a word to describe an environmental condition or physical element in English, the Chinese also use it in the context of the body. Not only does this scenario represent a fundamental issue of translation but also presents two diverging relationships between the body and the surrounding environment.

123

124

65 beats per minute 62 breaths

125

07 [Gesture 3]

qi/air Emotion

Ambience

SITE: Observatory, Can Lis Technology: Pulse sensor Body: Lungs and Heart Actions: Breathing and walking

Pulse readings

126

Film construction

Breath

Summary

SUMMARY ~~ GESTURE 3 is situated in the observatory and explores the reciprocities between two expressions of emotion: the pulse and breath, both of which are invisible responses to the surrounding enviroment. These two forms of expression, are documented and materialised through an installation and film, exploring the potential of using emotion as a material to explore ambience.

The installation creates a scenario where the overall ambience/result is defined by these two forms of expression, demonstrating a process where body, technology and place work collaboratively to generate an image , in line with the Chinese concept of Qi. The speed and movement of salt particles are defined by the live breath rate of the body during the filming process, while the rhythm/frame rate of the projected light is based on the recorded pulse rate experienced in Can Lis, therefore defining when the particles are illuminated and visible to the camera.

127

2 3

0 Introduction

Transition to Gesture 3 128

1 Gesture 1: Cave

2 Gesture 2: Living room

3 Gesture 3: Observatory

C a m er a p ath 129

Film stills : Transition to Gesture 3 130

131

132

i) THE READINGS ~

GESTURE 3 is situated in the observatory and explores the reciprocities between two expressions of emotion: the pulse and breath, both of which are invisible responses to the surrounding enviroment.

These two forms of expression, are documented and materialised through an installation and film, exploring the potential of using emotion as a material to explore ambience.

133

Body: Breath We are not normally conscious of our breath, but it is an involuntary exchange between us and the environment we are in. These changes in breath rate can be easily felt by the body either in the chest or stomach.

134

07: Gesture 3

Film still Cave

Technology: Pulse sensor The pulse sensor detects subtle movements in the skin, recorded as BPM (Beats per minute) which cannot be accurately captured or seen by the human eye - these readings are stored and documented as finite set of data.

135

Plan of Observatory

Body: Breath As the two characters progress further away from the house towards the edge of the cliff, from a semi enclosed space to an entirely exposed space, their emotions transition from a sense of safety to a sense of fear, where both pulse and breathing rates increase. For each breath a singular line is drawn. As the body draws in air, a line is drawn upwards , and back down when the air is drawn back out.

136

Time

137

Technology: Pulse sensor Using an arduino set up and pulse sensor connected to a laptop, pulse readings were plotted onto a graph and exported as raw data.

138

139

5

4

2 3

1.

2.

3.

4.

5.

Arduino board Pulse sensor Relay LED 12V Powerpoint

Tests: Pulse responsive light 140

1

*/ // int int int

Variables PulseSensorPurplePin = 0; // Pulse Sensor PURPLE WIRE connected to ANALOG PIN 0 LED13 = 13; // The on-board Arduion LED relayPin = 10; // Relay signal pin

int Signal; int Threshold = 550;

// holds the incoming raw data. Signal value can range from 0-1024 // Determine which Signal to “count as a beat”, and which to ingore.

void setup() { pinMode(LED13,OUTPUT); // pin that will blink to your heartbeat! pinMode(relayPin,OUTPUT); Serial.begin(9600); // Set’s up Serial Communication at certain speed. } // The Main Loop Function void loop() { Signal = analogRead(PulseSensorPurplePin); // Read the PulseSensor’s value. // Assign this value to the “Signal” variable.

Serial.println(Signal);

// Send the Signal value to Serial Plotter.

if(Signal > Threshold) { digitalWrite(LED13, HIGH); digitalWrite(relayPin,HIGH); } else { digitalWrite(LED13, LOW); digitalWrite(relayPin,LOW); } delay(10); }

Code for set up

141

142

ii) THE SCENARIO

143

Film stills Duration: 1:15 minutes 144

145

The scenario The installation creates a scenario where the overall ambience/result is defined by these two forms of expression, demonstrating a process where body, technology and place work collaboratively to generate an image , in line with the Chinese concept of Qi. The speed and movement of salt particles are defined by the live breath rate of the body during the filming process, while the rhythm/frame rate of the projected light is based on the recorded pulse rate experienced in Can Lis, therefore defining when the particles are illuminated and visible to the camera.

Both of these processes are directly related to time, place and experience.

This is similar to the perception of the cosmos, where the visibility of the stars is defined by the quality of air/ atmosphere.

146

06: Gesture 3

7 9

3 11

6 2 1 4

10

12

5

8

Plan of set up

1.

2.

3.

4.

5.

6.

Projector Altered fog machine Fog machine remote control Power point Laptop Table

7. MDF Backdrop 1 8. MDF Backdrop 2 9. Fabric backdrop 10. Camera position 1: Close up 11. Actor 12. Camera assistant

147

Breath rate on 16th June 2020, Grymsdyke Farm, UK

Breath 148

06: Gesture3

3 5

4

2

1

1.

2.

3.

4.

5.

6.

Fog Machine Metal connector Washing machine pipe Metal connector Silicon seal Silicon tube

A fog machine was altered to allow the speed of the fog to be controlled by my breath. The fog was infused with salt and flour particles, mimicking the materiality of the site.

149

Pulse to rhythm/frame rate 150

06: Gesture3

IBI (time intervals between beats)

Using the pulse readings, the flickering of the projector was determined by the time intervals between the individual beats.

151

Grymsdyke Farm

The set 152

06: Gesture3

Grymsdyke Farm

153

APPEN

~

154

NDIX

~

155

150 app

Light paths

2

1500 approx

Indicative forms and scale

Pl a n

Sec ti o n

1500 approx

1 1

Light paths

2

2

1500 approx

3

Plywood supports where necessary

4

Previous design development

Sec ti o n

156

Pl a n v i ew

Per s pective v iew

x

iew

Silver /copper coating to amplify

1 1

3

2 3

Plywood supports where necessary 4

4

Per spective view

Init a l sketc h o f pro po sal G e st ur e 1 157

(R)

(R)

Speaker playing recording (L)

Speaker playing recording (L)

1700 approx

(L)

(R)

(L)

(R)

1700 approx (L)

Indicative forms and scale

(L)

Indicative forms and scale

Sec ti o n

Sec ti o n

Previous design development

158

Pl a n v iew

Pl a n v iew

Speaker playing recording (R)

Carved forms act as reflectors (R)

Speaker playing recording (L)

An existing room (?) filled with earth/sand

(L)

Pla n view

Per s p ec ti ve v i ew

I ni t a l s ket c h o f p ro p o s al G e st ure 2

159

G-CODE (geometric code) is a language that tells the CNC how to perform. Each code is assigned with a specific instruction that is performed by the machine, for example, the G1 command tells the print head to move at a specified speed from the current position to the coordinated specified by the G-code command.

The movements of the machine are based on three basic types of motion.

Clockwise /anticlockwise

End position (x,y)

End position (x,y)

I= X-offset to center Start position (x,y)

Fastest speed

Start position (x,y)

At feed rate

Center point

G0

G1

G2 or G3

[Rapid move]

[Feed move]

[Circular move]:

a linear move to an XYZ position as fast as possible

a linear move to an XYZ position at a defined feed rate (defined by F)

a circular move (defined by I + J)

Research into G-CODE 160

J= Y-offset to center

The cnc will follow a series the instructions set out by the G-CODE and perform the actions line by line.

A typical line of code is formed as following:

[ G## X## Y## Z## F## ]

Motion/ function

X-axis position

Y-axis position

Z-axis position

Feed rate

M-COD E The line of a program might also include m-codes, which are generally codes that tell a machine how to perform an action. While not guaranteed to be the same across machines. /// Models are machined in 2 steps, roughing and finishing.

Roughing: coursely removes excess material layer by layer Finishing creates the final end result Use CAM software for toolpaths

161

Great Mosque of Djenne, Mali Due to the mud construction, the mosque requires regular maintenence due to the erosion and weathering of the surface throughout the year. The local community run an annual festival that celebrates this event, that involves maintaining and restoring the surfaces using their own hands, where gestural marks and traces remain visible and embedded in its form. This generates a relationship that is reciprocal.

162

163

BIBLIOG

~

164

GRAPHY

~

165

166

P r i m a r y re f e re n c e s : A b r a m , D av i d , T h e S p e l l o f t h e S e n s u o u s : Pe rc e p t i o n a n d L a n g u a g e i n a M o re - t h a n - H u m a n Wo rl d ( N e w Yo rk , 1 9 9 6 ) E d w a rd T. H a l l , T h e H i d d e n D i m e n s i o n ( A n ch o r, 1 9 6 6 ) Fe h e r, M i ch e l , Fr a g m e n t s fo r a H i s t o r y o f t h e H u m a n B o dy, 2 ( N e w Yo rk : C a m b r i d g e, M a s s. d i s t r i bu t e d by M I T P re s s, 1 9 8 9 )

Fr a m p t o n , Ke n n e t h , ‘ Jo r n U t zo n : Tr a n s c u l t u r a l Fo r m a n d Te c t o n i c M e t a p h o r ’ , 1 9 9 6 < h t t p s : / / d e z i g n a rk . c o m / bl o g / k e n n e t h - f r a m p t o n - j o r n - u t zo n - t r a n s c u l t u r a l - fo r m - a n d - t e c t o n i c - m e t a p h o r / >

K r u s e, C h r i s t i a n e, a n d A n t j e M a j e w s k i , Z i r a n / N at u re : A r t , N at u re, a n d E t h i c s ( S t e r n b e rg P re s s, 2 0 1 5 ) Ku r i ya m a , S h i g e h i s a , T h e E x p re s s i ve n e s s o f t h e B o dy : A n d t h e D i ve rg e n c e o f G re e k a n d C h i n e s e M e d i c i n e ( N e w Yo rk : Z o n e B o o k s, 2 0 0 2 ) L e ro i - G o u r h a n , A n d r é , G e s t u re a n d S p e e ch , 1 9 1 1 M e rl e a u - Po n t y, M a u r i c e, P h e n o m e n o l o g y o f Pe rc e p t i o n ( L o n d o n : Ro u t l e d g e, 2 0 1 3 ) Pa l l a s m a a , Ju h a n i , Ju h a n i Pa l l a s m a a I n t e r v i e w : O n J ø r n U t zo n , 2 0 1 8 < h t t p s : / / w w w. yo u t u b e. c o m / w at ch ? v = w z N TA q N d z H 8 & l i s t = P L 0 2 o I W j T S LG r C K I S 8 x v Ry m L y f _ D p D q a i C & i n d ex = 6 6 & t = 0 s > R . B r ya n t , L ev i , O n t o - C a r t o g r a p hy : A n O n t o l o g y o f M a ch i n e s a n d M e d i a ( E d i n bu rg h U n i ve r s i t y P re s s L t d , 2 0 1 4 ) Ro d aw ay, Pa u l , S e n s u o u s G e o g r a p h i e s : B o dy, S e n s e, a n d P l a c e ( L o n d o n : Ro u t l e d g e, 2 0 1 1 ) Ro t h e n b e rg, D av i d , H a n d ’s E n d : Te ch n o l o g y a n d t h e L i m i t s o f N at u re, 1 9 9 5 S e re m e t a k i s, N a d i a . , T h e S e n s e s S t i l l : Pe rc e p t i o n a n d M e m o r y a s M at e r i a l C u l t u re i n M o d e r n i t y ( C h i c a g o : T h e U n i ve r s i t y o f C h i c a g o P re s s, 1 9 9 6 )

S e c o n d a r y re f e re n c e s : B e n t h i e n , C l a u d i a , S k i n : O n t h e C u l t u r a l B o rd e r b e t we e n S e l f a n d t h e Wo rl d , E u ro p e a n Pe r s p e c t i ve s ( N e w Yo rk ; C h i ch e s t e r : C o l u m b i a U n i ve r s i t y P re s s, 2 0 0 2 ) B a rk e r, Je n n i f e r M . , T h e Ta c t i l e E ye : To u ch a n d t h e C i n e m at i c E x p e r i e n c e.

C l e m e n t , G i l l e s, T h e P l a n e t a r y G a rd e n a n d O t h e r Wr i t i n g s, 2 0 1 5 C o n n o r, S t e p h e n , T h e B o o k o f S k i n ( Re a k t i o n B o o k s, 2 0 0 9 ) C o o l e, D i a n a , N e w

167

168

169