On Ceding Space To Non-Human Systems

The fallacies of quantifed thinking and

its im plic a for ns tio e nt u r diminished moral agency in 21st c r yu

ba

nis m

On Ceding Space To Non-Human Systems Benjamin Dresner-Reynolds

2

2

BENJAMIN DRESNER-REYNOLDS TUTOR: MARK GARCIA Architectural Thesis MArch ARCT1060 2018

1

2

This thesis is accompanied by a sound art installation.

A dial lies on the face of the installation. When turned all the way to the left, a text-tospeech voice reads the original essay out loud. As the dial is turned to the right, ‘AI’ generated content on the same subject begins to creep into the text, merging with the original writing to a greater and greater extent. The automated content merges with the text mid-sentence, at junctures calculated by grammatical rules. When pushed fully clockwise, the thesis is entirely ‘AI’ generated, and will carry on generating content indefinitely, albeit often in quite an absurd fashion.

The text is generated by a model of Marcov chains building sentences from a corpus of the source material and reference texts used to write the original thesis.

Made possible with the help, support and coding skills of Jonathan Davies.

2

2

TABLE OF CONTENTS

RESEARCH METHOD STATEMENT...........................................................5 ABSTRACT..................................................................................................12 INTRODUCTION..........................................................................................13 QUALITATIVE CONTROL...........................................................................15 Big data & machine learning FIELD TESTS IN FATALITY........................................................................18 Driverless cars & the trolley problem REMOTE DYING..........................................................................................23 The virtues of killer robots AETHERNET................................................................................................26 Digital borders EMERGENT SYSTEMS...............................................................................29 Second-order cybernetics NETWORK CONNECTIONS........................................................................32 Horizontality in the smart city SOCIAL MEDIATION....................................................................................36 Manipulation & mental ergonomics (CHOICE) ARCHITECTURE.........................................................................39 Construction, politics & process PARAMETRIC INTERFACES.......................................................................42 Symbols, simulation & the market CONCLUSION..............................................................................................46 BIBLIOGRAPHY...........................................................................................48

3

8

9

10

11

ON CEDING SPACE TO NON-HUMAN SYSTEMS for diminished moral agency in 21st century urbanism

ABSTRACT casts a loose but useful net over a phenomenon present across a widening array of human undertakings, in which we attempt ever greater epistemological control over our society and environment. Models of machine learning have expanded the role of data endowing such technology with the means to act in ways which have growing of autonomy to machines that must operate freely in urban space according programming them. The implications for human moral agency are addressed, as are concerns about the use of such technology for disciplinary state power.

life in the 21st century city. This is enacted through a consideration of Masmart city and its roots in in second-order cybernetics, and how this emerging condition may limit our means to freely establish our own values. Finally, these considerations are applied the built environment, evaluating the effects of da-

12

INTRODUCTION

insight or a new dimension of commercial revenue. New devices and services worse. The informational revelations of the network appeal to the idea that Through the advent of

As such, we are applying technology to areas of life in which judgement is a matter not only of weighing up facts, but of philosophical interpretation. There much to the technology itself as to those who design and regulate it - in striving to solve problems beyond our cognitive capability we have developed algorithmic tools whose behaviours we cannot always understand. An expanding range of societal functions and human roles are supplemented or replaced by such technologies and such convenience often comes at the expense of some degree of autonomy. This paper examines the developments which challenge our moral agency in the matters of shelter and spatial freedom which are pertinent to the human experience of the built environment.

13

Figure 1

14

QUALITATIVE CONTROL Big Data & Machine Learning

broadly describes a mentality that strives to manage every We exist in an era of growing connectivity, where content can be gleamed

the intoxicating lure of

is feeding a solutionist notion;

ally talking about the computational analysis of the kind of measurable infor-

intensive, and usually cannot be computationally assessed in the same way as numeric content.

-

Even in government intelligence, where one must assume that ample technological resources are on offer, we see datasets available today: -

When it comes to gaining interpretive insight, more data can actually be counWinton Professor of the Public Understanding of Risk at Cambridge University � is an area of computer science transforming many aspects of our society, from search engine optimisation to economic growth predictions. It 15

allows us to create algorithms that can analyse these huge, unstructured sets instances where the algorithm only needs to recognise patterns that emerge from the data in order to achieve the desired results - understanding becomes 2013, a now classic example of the potential pitfalls of machine learning. When -

-

is allowing software to achieve competence in areas previously thought to be reserved only for humans by utilising multiple layers of neural networks - computational learning systems inspired by omous vehicles that we have seen in recent years. It allows for the advent of machines that operate on the basis of environmental information alone, acting in the physical world in real time with no human guidance or oversight. This is

as .

These autonomous technologies have created a new dilemma, one that relates as much to urbanism as it does to governance and computer science: if machine intelligences acting independently in the physical world are to behave in ways that are as safe for humans as possible, they will have to be designed to act in accordance with some set of moral values. On February 12th, 2018, The New York Times reported that Harvard and MIT were introducing a joint course been forced into a positivist moral position by the highly divisive philosophical challenge of having to computationally codifying ethics.

16

Figure 2: Lidar Scan, London

17

FIELD TESTS IN FATALITY Driverless cars & the trolley problem

The Trolley Problem is an ethical thought experiment, the modern version of which is attributed to a 1967 paper by Philippa Foot. It has many different ver-

There are many variants that explore these issues of morality and agency in

this problem to life, as manufacturers, programmers and lawmakers begin to consider how these vehicles should behave in an imminent crash situation.

ing that this is the kind of car they would like others to purchase.

Just as in the classic Trolley Problem, when faced with the stark mathematics, the majority go with a utilitarian response. However, Bonnefon et al. themselves. They state that the following three groups are in contention to resolve these ethics dilemmas: drivers and their communities, governments, and manufacturers themselves . dismiss the idea of a system of ethics emerging from some calculated aggregate of social values as unrealistic:

Etzioni

18

Of course, not all communities share the same ideals, and the thought of multiple value sets working together seems problematic, as does the idea of each

legally acknowledged moral relativism.

Many surveyed were strongly opposed to government deciding such matters . Nevertheless, both the European Union and Germany recourse in the result of machine errors and offering the right to an explanation,

should be settled on a case-by-case basis:

-

-

dilemma. She suggests that strategies for addressing these issues are still taking shape.

-

-

Furthermore, there is the issue of who is held accountable in the event of an 19

-

extreme to suggest that future iterations of these cameras, sensors and other recognition technologies could potentially be utilised to determine crash decitheir worth accordingly, or that the cars themselves could be networked in order to confer with each other in the moments before a collision prior to making WNYC Studios demonstrates that these are possibilities to which government is already apprehensive.

While legislation is an important safeguard, it cannot cover every eventuality.

Illah Nourbakhsh, professor of robotics at Carnegie Mellon, predicts unfore-

-

forever change the nature of our cities. They will be trained and guided by the presence of the bystanders, buildings and infrastructure, which in turn will be implicated in their decision making processes. Designed as they are to learn from their surroundings in order to improve their means of engaging with their will add a new algorithmic layer to the fabric of the city itself.

20

The UK government commissioned a a 2017 report from Catapult transport Taxonomy of Scenarios for abnormal situations where human oversight might be necessary. On this, Tay-

or even for allowing humans to drive at all, will become ethically indefensible by huge numbers, the advent of autonomous cars raises issues of morality io, algorithms, programmers and legislators will end up making calculated, premeditated decisions about who will live or die, removing all autonomy in this matter from those directly involved.

Etzioni and Etzioni

.� Such circumstances would mean not only a loss of individual agency to whomever

-

21

Figure 3: Dronestagram

22

REMOTE DYING The virtues of killer robots

The increased military use of unmanned, remote operated drones have been tional air strikes or ground invasions, theoretically minimising the risk of civilenabling a military force to perform operations without risking the lives of their troops. However, their use remains highly controversial, and a contested issue in legal circles. Disagreeing with drone proponents, Waseem Ahmad Qureshi highlights several concerns:

-

The military application of big data ensures that some errors will cost innocent lives; indeed, Andrea Prasow of Human Rights Watch maintains that drones

-

exaggerated trust in data impinges on the capacity of politicians, military personnel and voters to make informed decisions on these matters. Additionally, -

with the human responsibility behind attacks:

-

temporary imagination. On one hand, it is an example of a new and hi tech weaponry, on the other it is projected as a modern instantiation of mystical ar23

-

It world through sensors, cameras and so forth, that gives it this metaphysical ence of its environment, and can thus be used to forcibly act upon it.

inherent in conducting military airstrikes. Contrary to autonomous vehicles, which could be said to render a certain impotence over car owners that are used to being in the driver s seat, the drone s articulation of perennial human for its operators, rendering mortal soldiers invulnerable in previously life-threatening combat situations. However, it is worth mentioning that levels of PTSD amongst drone operators are as high as amongst traditional service personnel

-

There is often an unnerved human response to the uncanny precision and purposefulness of these robotic technologies; perhaps partly due to the fact that these abilities represent the possibility of human capabilities being entirely eclipsed, or human beings being existentially threatened. The very idea of LAWs conjures up cliche this problem of tremendous ethical and legal implication only barely remains a Ackerman

as an ethical and legislative challenge.

Drones have gained particular notoriety for their deployment by the US military being transformative to . The drone has become the weapon of choice for instances of breaching national sovereignty. Such technology creates new spatial dynamics not only for international power relations, but also for new methods of internal disciplinary state control, affecting individual capacity for free thought and action.

24

Figure 4: Autonomous Trap 001

-

25

AETHERNET Digital borders Aside from the border transgressions of drone strikes, there are other ways in which technology is effecting our consideration of national boundaries. Bridle � status of locations such as airports are used tic government agencies with a different legal interpretation can readily detain lation governing spatial demarcations are not set in stone, and are in fact only Online connectivity allows these wills new means of remote spatial behaviour. In order to determine whether migrants attempting to enter Australia are truly travelling from a place at which they are at risk, and to avoid committing the act using networked communications to assess the status of migrants before they

-

crossing the border, assessing their movements, speech and facial cues for interaction with automated systems or remote technology removes the means

nervous twitch, or the raised pitch of a frightened voice, algorithmically mistaken for deceit, are errors one could imagine being made by a rudimentary AI, but perhaps could be easily contextualised by a human observer. Effectively automated administrative systems may assume a certain infallibility or demonstrate a lack of accountability, whilst recourse for human appeal could prove problematic in time-critical circumstances.

-

26

The advent of blockchain based has raised similar fears about spatial restriction supervised by autonomous technology These enable contractual agreements to be drawn up and automatically enacted algorithmically, could result in situations where, for example, failure to keep up with mortgage

raised by Gilles Deleuze and Felix Guattari in the early 1990s about new models of control. This will begin by looking at second-order cybernetic thought hyper-connected models of data interaction that have been termed the

27

28

EMERGENT SYSTEMS Second-Order Cybernetics

tions, but it can be broadly thought of as a language for describing self correctachieving its goals. For example, in humans, this information may take the form of reading, conversation Heinz von Foerster and systems as closed, observable phenomena that are naturally drawn to harmony and balance; it

systems as closed, were able to be probabilistic, SOC argued that

stems from the logic of cybernetics, critical to which However, this speaks pretation that

It follows from this that the city, rather than being an enclosed web of interlinked apparatus that could be planned perfectly with the right tools of measurement and means of governance, is in fact something of a black box itself. However, he argues that this means neither that there exists a safeguard

Deleuze looks at the 29

contradictions in our epistemological attitudes towards data. Whilst his conlater extrapolated for the broader consideration of technological impacts upon moral agency.

-

30

Figure 6: Smart Bin

31

NETWORK CONDITIONS Horizontality in the The Smart City

-

that of linking , something that is truly becoming instantiated in the 21st century smart city, as we witness the explosion of myriad networked gadgets and infrastructure that has been labelled the

The idea of the smart city promises that by extending existing network techinsightful management of urban life; such ideas are regularly touted by its pro-

, representing a which outlined new modes of political governance that are complex, non-linear and horizontal in nature. but Deleuze argues that it is this very increase of interaction that enacts a sort of normative constraint.

-

is an expansion of which he gravitated towards as his thinking moved away from a prior focus on disciplinary models of power, citing 32

governmentality also draws a comparison with cybernetics: -

Whilst early cybernetics maintained parallels of thought with traditional ideas of control, in which a liberal subject may face top-down coercion through the and unpredictable transformed the liberal subject into -

eventually ended up informing new, indirect methods of state power, that uti-

of the smart city is incompatible or threatening to the informal nature of the -

support the notion that mass connectivity can substantiate horizontal modes of behaviour regulation that may then self-perpetuate via positive feedback.

-

KrivĂ˝ proposes the term

to describe this

subject themselves are conceived of as a simplistic, sensing, purposeful unit; -

The goal driven aspect of cybernetic logic implies the application of smart metrics to human intentions as well as to behavioural trends. The idea of trying 33

to measure, predict or effect human intentions based on data has been exem-

could be similarly utilised in the smart city across a vast range of social, political cultural and infrastructural areas of life; from the automated optimisation enforcement based on the live cross-referencing of crime hotspots with social or economic triggers.

idea that a totality of understanding and control of such complex human events is impossible:

-

Nonetheless, KrivĂ˝ claims that even the supposedly democratic proliferation of

. Whilst his focus is on the political power behind such methods of con-

tion platforms by which nodes in the smart city network organise.

34

Figure 7: Antisocial

35

SOCIAL MEDIATION Manipulation & mental ergonomics

Environmental-behavioural control can be witnessed today in many areas of online life, particularly through social media. One could cite the example of conform to particular beauty norms perpetuated by social networks like Insion trends develop, aesthetic standards crystallise; as the network grows, the back into the network, and this data feeds marketing analysis and targeted advertising by third parties across society, etcetera. Some will claim that this

.�

The hard-coding of autonomous vehicles ethics removes moral agency in a direct way. Contrastingly, the normative effects of social media occur on platforms with a global reach of millions, and have the effect of standardising certain values across society, limiting our capacity for behavioural autonomy by making it harder to deviate from social or group norms. Network science uses the term to describes the tendency of similar nodes to cluster together; there are concerns that this occurrence on social media has been a factor in the increased political polarisation in the United States in be similarly responsible for the growth of political movements online :

The Deleuzian notion of the citizen, conditioned by the environmental-behavioural control of the city, speaks to a real concern over the power of netAside from such content related nudging of our actions, the design of smartphone capacity for independent behaviour. Tristan Harris, former Design Ethicist at 36

Google, has established Time Well Spent, an advocacy group calling for pubcompanies as they battle against each other for our clicks and screen time: “ -

This is the idea of Our experience of interfacing with technology, whether though an Apple or Google smartphone operating system or that online world, but in fact it has been designed; we are navigating through a predetermined list of curated choices. Harris argues for a new design ethos of mental ergonomics, -

to reclaim control of our technological experience is a pertinent one. Initiatives will just result in new, concrete forms

37

Figure 8

38

(CHOICE) ARCHITECTURE Construction, politics & process

exists in a different political condition, one which it is mostly unable to affect.

-

the humanitarian effects of construction are thus reserved for theory.

Just as the smartphone user must operate within the constraints of of an operagency is additionally constrained by the tools at its disposal. Design programs are artefacts of the conditions of control that Deleuze and Foucault warn of, which practitioners in the built environment cannot help but perpetuate: -

Traditionally, construction projects are developed collaboratively across sevand as a result, enables networked collaboration on complex, parametric projects - enables contractors can all share information in the same data-rich 3D model, allowing for more predictable and reliable design outcomes. As BIM, digital fabrication 39

and construction process becomes more streamlined and less convoluted

telligence�, compounding the situation described by Shvartzberg - a situation design process and architecture grows increasingly consumer focussed

believes that architecture could go in the direction of product lifecycle manwhere, due to means of easily resolving designs at the

-

-

The growth of BIM, IoT and smart materials may create an architecture in which tous elements, with user experience data both informing a design brief and

�, that places -

The consumer is free to independently choose whichever car manufacturer where the customer and the user are rarely one and the same; most users of buildings are subjects of that space, the customer is the client. In asking whether current procurement methods will be replaced in an emerging smart design culture that can harness the informative power of data, Phillip G Bernstein,

40

Architectural user data will have value in informing design, but will not provide the architect or the ordinary user of public or commercial space any means of affecting the key aspects of user experience in the city. These are not sold experiences like that of a luxury car, infrastructure - which stand as the vital role of the built environment.

These changes in the industry would reinforce the present condition of a depoliticised architecture that is unable to address the problems which it must

-

41

42

PARAMETRIC INTERFACES Symbols, simulation & the market

built environment speak to the same notions of horizontally that KrivĂ˝ applies to the smart city. These tools are modes of , an idea that origiand since the 1990s has generally been used to describe algorithmically intelligent architecture and the software used ture also stem from SOC. Cybernetician Gordon Pask directly inspired architects Paul Coates and John Frazer to theorise on autonomous, distributed spa-

of a liberated user, working in symbiosis with an adaptive environment:

-

tion of BIM and IoT, his original notion of such a built environment was one

Schumacher,

Whilst

Schumacher claims that the downfall of state-backed modernist planning and

ideology must be combined with a new stylistic architectural coherence based on the parametric logic of scripted tools. This can produce a new urban para43

digm of bottom-up, algorithmic spatial order, forging cities via the self-regulational difference becomes theoretically possible. Carpo speaks to the power of contemporary parametric design tools to transcend post-Fordist construction:

Schumacher claims that the built environment can achieve this social role -

ulations could be made applicable via

-

termining social function, does so through a simulacra that claims to represent human behaviour, but in fact reduces the user to a goal driven, economised unit. Claiming ownership of the citizen, he presupposes their voice through an assumption of universal market truths; they can thus be used to validate his formal language. Rather than a conditioned node of the city, they are able to

presented us with a theoretical vision of the city in which any human experience outside of the project brief is unnecessary, and thus entirely stripped away from the calculation. This device presupposes the viability of market logurban life.

-

data map that stands as the only acknowledgement of valued human activity.

Just as our capacities for independent thought and moral action are restricted by normative conditioning of the network, a consumer-led, data focussed 44

model of design could entrench design values in data, enabling a market-led nudge of smart architecture that pulls agency away from both the architect and the user, establishing standards that proliferate regardless of their merits, or

world mould our experience of it, the software tools of architectural practice ty will be the means of establishing such conditions.

45

CONCLUSION

-

forces that drive human behaviour, the subtleties of experience that make up the internal world from which these desires and actions emerge cannot be truly known or understood by computation. These approaches must invariably make

become ever more effective, this reduction of human teleology to data points will become the true representation of ourselves in the eyes of those reading the data. Just as the neoliberal assumption that experience can be evaluated by economic metrics fails to address aspects of existence that lie outside the

into database parameters.

measure. In pursuit of automated convenience, we may end up establishing immutable social and ethical norms that risk undermining the depths of philosophical thought which have endured and evolved over millennia. These are jective ethical truth; our contrasting beliefs result directly from the imperfect and diverse nature of the human experience. However, this same fallibility puts us at risk of the hubristic notion that through data we can either discover ultimate truths in earnest, or project the notion that we have done so. As those that ic strata of society, such eventualities are at risk of fundamentally impacting ordinary human agency in matters of ethics, values and spatial behaviour. nance, social planning, or even philosophical reasoning and understanding 46

omniscient technology would be a secession of control to a force we cannot comprehend, whose reasoning we cannot know. The mysterious methodologies of the deep learning processes that are transforming society are creating a situation where developers themselves increasingly lack a complete understanding of the modes of reasoning which they are creating. We have already nological capability for control over the decisions that govern our lives.

47

BIBLIOGRAPHY

forsocialhousing, [online] Available at: https:// architectsforsocialhousing.

alism of Levitation, Geographica Helvetica, 71, p. 319–329 Practice & Criticism, 17,p. 18–34 -

https://www.economist.com/news/asia/21524916-growing-controver-

mous Weapons in New Open Letter, IEEE Spectrum: Technology, Engineering, and Science News, [online] Available at: https://spectrum.ieee.org/automaton/robotics/military-robots/industry-urges-united-nations-to-ban-lethal-

Body-subject, Society, 22, p.58–81

& Society, 20, p.1–13

32, p. 251–266 48

tion, Architectural Design, 86, p.60–67. tonomous vehicles, Science, 356, p. 1573–1576 able at: https://www.theguardian.com/us-news/2018/jan/ 23/us-air-wars-trump

-

Digital Creativity in the UK – Medium, Intersections: Art and Digital Creativity in the UK, [online] Available at: https://medium.com/intersections-arts-anddigital-culture-in-the-uk/james-bridle-machine-learning-in-

ports, [online] Available at: http:creativetimereports.org/2014/11/10/james-bridle-electromagnetic-border-zone/

phone addiction in university students- A cross sectional study, Plos One,12, p1-14 ical Polarization? Evidence from Demographics. Proceedings of the National Academy of Sciences of the United States of America, 114, p. 10612-10617

of Choice, Brookings, [online] Available at: https://www.brookings.edu/ar-

49

tectural Design, 86, p. 24–29

Media, [online] Available at: https://www.theguardian.com/education/2013/

Facebook and Instagram appearance-focused activities and body image concerns in young women, Body Image, 23, p.183–187.

bridge

84, p. 14–23

in Combat Do, The New York Times, [online] Available at: http://www.nytimes. com/2013/02/23/us/drone-pilots-found-to-get-stress-disorders-much-as-

US election?, The Observer, Guardian News and Media, [online] Available at: https://www.theguardian.com/politics/2017/mar/04/nigel-oakes-cam-

ment of Urban Public Space: Universal Design for Intelligent Infrastructure, Journal of Urban Technology, 21, p. 21–38

Technology, 18, p. 149–156 ture, Automated and Connected Driving, Germany

50

fect, Oxford Review, 5

college de France, 1978-1979, Palgrave Macmillan.

sensing in the smart city. Environment and Planning D: Society and Space 32, p30–48.

Should Care, Tristan Harris, [online] Available at: http://www.tristanharris.com/2016/05/how-technology-hijacks-peoples-minds %E2%80%8A-%E2%80%8Afrom-a-magician-and-googles-design-ethicist/

Insider, Thrive Global, Thrive Global, [online] Available at: https://journal. thriveglobal.com/how-technology-hijacks-peoples-minds-from-a-magician-

Computation in Building Design, Architectural Design, 87, p. 74–81

and the society of control, Planning Theory, 17, p. 8–30

Google Flu: Traps in Big Data Analysis, Science, 343, p. 1203–1205 London, E. J. B.

Driverless Cars, BLDGBLOG, [online] Available at: http://www.bldgblog. com/2015/11/computational-romanticism-and-the-dream-life-of-driver-

51

Intelligent Buildings of the Future: Cyberaware, Deep Learning Powered, and Human Interacting, IEEE Industrial Electronics Magazine, 10, p. 32–49

double in a year, The Guardian, Guardian News and Media, [online] Available at: https://www.theguardian.com/global-development/ 2018/jan/08/civilian-

-

of Integrated Software Platforms in the Production of Architecture, Architectural Design, 87, p. 34–41

oric of U.S. Drone Imagery, Quarterly Journal of Speech, 101, p. 612–632

Dame Journal of International & Comparative Law, 7, p91-106

Parametric Semiology, Architectural Design, 86, p. 108–113

Urban Order, Architectural Design, 86, p. 114–123

Want to Address It, The New York Times, The New York Times, [online] Available at: https://www.nytimes.com/2018/02/12/business/computer-sci-

52

immaterial labor, the creative class, and the politics of design, Bloomsbury Academic

robots: Driverless vehicles, ethics, and the future of transport, Transportation Research Part C: Emerging Technologies, 86, p. 206–215 structed Architecture, Architectural Design, 86, p. 54–59. ernment policy. Email conversation with E. Taylor, 6th March 2018 ban Studies 51, p883–898.

the machine, New York: Wiley

-

http://www.radiolab.org/story/driverless-dilemma/

53