DRONESPHEREASSEMBLAGE
10 pamphlets on aerial architecture Simon Rabyniuk
02
Simon Rabyniuk s.rabyniuk@daniels.utoronto.ca 2019
[It] has become so difficult to layout the object of study, to put computer code, network architectures, user interfaces, and so forth, on the dissection table... Geert Lovink, Hermes on the Hudson
Assemblage
3
Prime Air, Amazon 5lb
Raven x1400, Drone Delivery Canada 25lb
Prototype, Boeing 500lb
Mule, Flytrex 5lb
VF-41, JD.com 11lb
10 20 30 40
RQ-7, AAI Corp 60lb
50 60 70 80 90 miles
100
Zips, Zipline 3.5lb
(Left) Drone Delivery Comparison: Range, Payload Weight, and Drop Site
Assembling_Drones
As social, technical and political nodes, drones — autonomous or otherwise — are constellations of parts that constitute multi-scalar assemblages. Said differently, the composition of parts that physically produce lift, the legal apparatuses that bound movement, the technical and algorithmic components enacting computer vision, all concurrently exist. These interactions and the composition of different parts become visible as you view them at different scales. Each scale presents a distinct spatial quality and thus aesthetic. This pamphlet, however incompletely, attempts to represent these part-to-whole relationships, as well as to explore them at different resolutions.
Assemblage
5
1. 1. 2.
3.
4. 1_SENSORS_NAVIGATION 1_SENSORS_PAYLOAD 2_SENSOR FUSION 3_COMPUTER VISION 4_TRANSMIT/RECEIVE 5_MOVEMENT
5.
Assembling_Parts
Component parts have relations of interiority, which means they appear fused, as if they were a single and whole thing.
COLLECT PIXELS VERTICES FRAGMENTS TrainingEDGES sets are monuments of human labour. They require people to view and semantically label thousands, if not more, of still or moving images.
-PROCESS ToPRE date this task isIMAGES not automated. Startups, BRIGHTNESS GEOMETRIC LOCALIZE servicing companies employing machine learning, either offshore the creation of these sets or employ a distributed workforce in model similar to Amazon’s “Mechanical Turk.” Offshore labeling factories have been setup in many places FEATURE EXTRACTION including one in Kibera, which is identified as the EDGES REGIONS CORNERS world’s largest slum, located in Nairobi, Kenya. Training sets are the foundation of machine learning algorithms. These algorithms operate as black boxes in which the specifics of any deciANALYSIS/DECSIONS/ACTION sions are not availableBUILDING for analysis. When they MOUNTAIN THUNDER RANGE CLOUD ways the only means of human act in unexpected intervention is modifying their training sets. Assemblage
7
1 2 3 4
5 1_ENVIRONMENT 2_DEVICE 3_SENSORS 4_NAVIGATION 5_HUMAN-IN-THE-LOOP
Assembling_Ensembles
An assemblage of assemblages, drones functions as an ensemble of parts which have relations of exteriority. This means that the parts (e.g. satellites, remote controls, human operators, etc.) act together, but remain expressed as separate and unique elements.
NAVIGATION NAVIGATION NAVIGATION
STORAGE
PASSAGE
STORAGE STORAGE
PASSAGE PASSAGE
Assemblage
9
ENVIRONMENT
Temperature, Precipitation, Humidity, etc.
Consideration for a drone’s reliability must evaluate its performance in different climactic conditions. Romanticism aestheticized extreme weather, representing the fragility of human life in the face of great storms. Sweltering heat, ice rain, or gale force winds, gateways to the sublime, equally leave their mark on Turner’s shipwrecks as they do the drone.
WEATHER Assemblage
11
ENVIRONMENT
Animal, or Machine
Birds, insects, buildings, drones, among other things each occupy and produce space. In this way these living, non-living, and non-living animate things constitute a broad set of interactions. Whether in motion, or not, these relationships within an environment are more complex than the mere possibility of physical collision.
THING Assemblage
13
ENVIRONMENT
Electro-Magnetic Spectrum
Spectrum is the trans-Atlantic shipping routes of the 21st century. Expansive, yet finite, it is the space in-between the port cities of Genoa and Tunis. Access and control of spectrum is auctioned off by state-based regulators to an ever dwindling number of companies. Current ideas for drone air traffic control are predicated on drones occupying both air and spectrum through flight and active perpetual transmission.
SPECTRUM Assemblage
15
DEVICE
Rotorcopter, Fixed Wing, or Hybrid
Drone is a generic term for a broad class of flying machines. Fixed wing, rotorcopter, or hybrid drones have distinct capabilities. The former have an advantage in the areas of endurance and distance, while the latter benefits from vertical takeoff and landing and finer maneuverability.
MORPHOLOGY Assemblage
17
DEVICE
Lift, Pitch and Yaw
Within the race to master poweredflight the Wright Brothers were said to succeed by addressing multiple aspects of flight as a set of related problems. (e.g. takeoff/landing protocols, lift, propulsion, and steering).
.
MECHANICAL Assemblage
19
DEVICE
LIDAR, Radar, Visual, Etc.
The act of humans’ “outtering” capacities into machines involves finding a means to replicate, by other means, actions previously performed by us. Early gyroscope’s righted a plane allowing for its direction to become remotely controlled.
SENSORS Assemblage
21
DEVICE
LIDAR, Radar, Visual, Etc.
Autonomous vehicles employ a broad suite of sensors in which to locate themselves, within their specific environments, in and amongst a world of other things. These sensors describe space and objects distinctly from each other through their different types of inputs. Such a breadth of approaches for sensing the world is required given the variety of conditions that these vehicles negotiate.
SENSOR FUSION Assemblage
23
DEVICE
Graphics Processing Units
Price reduction for GPUs, along with their increasing processing capacity, are opening a wider and more mobile range of uses to machine learning algorithms. This facilitates the “outering� of situational decision making from human to computer and presents one of the cornerstones for automating flight.
GPU Assemblage
25
DEVICE
Training Set
Machine learning training sets are monumental feats of human labour. Creating one involves the semantic labeling of imagery. This involves processing thousands or even millions of images by tracing the edges of figures and classifying them (e.g. bus, human, bird, rock, etc.). Training sets establish baselines for a neural network’s interpretion of the world and thus how these machines take action.
TRAINING SET Assemblage
27
DEVICE
Transmitter, Transceiver, Airband
Applying alternating current to antenna induces waveforms in air which then transmit information over a distance. Spectrum is closely regulated to avoid interference. Aviation communication is broken into a lower set of frequencies for navigational aids, and a higher set of frequencies for voice communication. UAS traffic will require spectrum of its own.
TRANSMISSION Assemblage
29
DEVICE
INTERNAL / EXTERNAL
Drone’s log sensor input and geospatial information tracking its flight path. This information transmits in real time and for storage in multiple places including remote servers, with the pilot, and with the drone itself. For this reason hobbyist drone flight, by different countries’ military personnel, has been banned as these birds eye views make sensitive sites visible.
STORAGE Assemblage
31
SENSOR
Camera
The recording, storing and transmitting of images has moved from an act of chemistry in early photography to one of computation. Lenses still focus light, reflecting off of a subject, onto a receiving surface. However, visible optics have disappeared into a “black hole of circuits,� which is to say sensors recording the visible spectrum enact single processing.
VISIBLE SPECTRUM Kittler, Friedrich. Optical Media. pg 225
Assemblage
33
SENSOR
IR Thermography
Infra red is a non-visible range on the electromagnetic spectrum. Infrared sensors detect thermal radiation or heat. Everything above 0 degrees kelvin emits thermal radiation. Infrared sensors translate the infrared image into a radiometric one which allows for temperature values to be interpreted.
INFRARED Assemblage
35
SENSOR
Light Detection and Ranging
LiDAR is an active form of sensing. It emits light in the near infrared spectrum and measures the time of flight for it to reflect back off of a surface. The pulse system can emits 10,000 to 15,000 beams/second. Post processing can produce an environmental computer model, as well as analyze or remove elements such a as tree canopy.
LiDAR Assemblage
37
SENSOR
Multispectral Camera
Materials can be identified by their unique spectral signatures through recording differences in reflectance and emittance of wavelengths. For remote sensing uses drone payloads can include multispectral cameras. Through a rotating set of filters multiple images of a landscape are captured. Analysis is reliant on machine learning algorithms for material identification.
MULTISPECTRAL Assemblage
39
SENSOR
Hyperspectral Camera
Objects have distinct spectral signatures which reveal their material composition. This type of analysis is available by recording the full spectrum of electro-magnetic waves reflecting off of objects. Three dimensional representations, known as “data cubes,� present hyperspectral data. It allows acute analysis as a full spectrograph is available for each pixel in the data cube.
HYPERSPECTRAL Assemblage
41
NAVIGATION
Ground Control Point
In remote sensing applications, ground control points, which are placed by humans at known locations, serve as registration marks within a landscape. When these markers are captured in remote sensing images they provide a means for algorithms to remap sensor input into a known geo-spatial frame of reference.
GROUND CONTROL Assemblage
43
NAVIGATION
Air Traffic Control
Air traffic Controllers manage private, commercial and military flight from ground-based stations. Controllers issue directives to pilots applying air space separation rules towards preventing collisions. Current tests are verifying whether drone traffic management could minimize human oversight and rely on a situational rezoning of airspace based on emergent conditions and emergencies.
ATC Assemblage
45
100 AGL 10 KNOTS
100 AGL 10 KNOTS
Airspace User GWI588
100 AGL 10 KNOTS
Airspace User NMM211
100 AGL 10 KNOTS
4000 AGL
60 KNOTS
NAVIGATION
19000 AGL 400 KNOTS
19000 AGL 400 KNOTS
4000 AGL 60 KNOTS
Airspace User LJW284
100 AGL 10 KNOTS
4000 AGL
60 KNOTS
Airspace User CXZ644
Automatic Dependent Surveillance-Broadcast
2020—Air traffic control systems transition from ground-based radar and navigational aids to precise tracking, of each airspace user, via satellite. ADS-B broadcasts the location, altitude and velocity of each airspace user to every other one. In doing so it distributes information previously only seen by air traffic controllers.
ADS-B Assemblage
47
NAVIGATION
Radio Detection And Ranging
An active form of sensing that involves transmitting and receiving signals as two distinct phases of operation. It reached wide deployment during the Second World War. Emitting electromagnetic waves, it identifies objects in its surroundings by measuring the deformation in returning waves bouncing off of things. It still provides the backbone for air traffic control and may play a role in low-altitude flight.
RADAR Assemblage
49
NAVIGATION
Detect and Avoid
Detect and avoid is the capacity for a vehicle to sense its surroundings and to react to avoid hazards such as other vehicles, terrain, bad weather, groundbased events, and other air hazards. A vehicle might alert its remote pilot, or enact maneuvers on its own. Within these sensor-algorithm assemblages a key question remains as to the how risk will be evaluated across multiple actors.
DAA Assemblage
51
NAVIGATION
Differential Global Positioning System
GPS locates a receiver within a known coordinate system. It does so by calculating the travel time of signals from multiple satellites to the receiver and then triangulating a position in relation to those satellites. Many factors reduce the accuracy and precision from this form of localization. DPGS uses ground-based transmitters, with known locations, to provide real-time positional corrections to GPS signals.
DIFFERENTIAL GPS Assemblage
53
HUMAN-IN-THE-LOOP
Norbert Wiener’s “antiaircraft predictor” analyzed the flight pattern of WW2 pilots and predicted their forthcoming positions displaying this information to a person. Its invention helped Wiener formulate a diagram of the human mind in which it acts in response to black box controlled feedback loops. Drone pilots exist in this lineage, acting in ensemble with ergonomic chairs, screens, and interfaces.
PILOT Galison, Peter. “The Ontology of the Enemy: Norbert Wiener and the Cybernetic Vision,” in Critical Inquiry, Vol. 21, No. 1 (Autumn, 1994), pp. 228-266.
Assemblage
55
HUMAN-IN-THE-LOOP
US Drones operating in conflict zones have upwards of 192 personnel piloting them, providing imagery and legal analysis, and wider strategic oversight. Domestic drone use seems to suggest a reversal of this “many-to-one” relationship, with drones set to act automatically by following preset paths. In this scheme, one person may oversee many drones.
PERSONNEL Gregory, Derek. “Drone Geographies,” in Radical Philosophy 183, Jan/Feb 2014. Pg7-19.
Assemblage
57
HUMAN-IN-THE-LOOP
In occupying cities drones alter transform urban aesthetics. A user study reports that bystanders perceive drones as “spiders in the sky,” and that they instill uneasiness given uncertainty about what they are recording and who they might be associated with. These concerns are not currently addressed in current regulatory research.
BYSTANDER Chang, Victoria, Chundury, Pramod, and Chetty, Marshini. “ ‘Spiders in the Sky’: User Perceptions of Drones, Privacy and Security,” Conference Paper · May 2017.
Assemblage
59
HUMAN-IN-THE-LOOP
“[The] harmless citizen of postindustrial democracies [e.g. the Bloom]... readily does everything that he is asked to do, inasmuch as he leaves his everyday gestures and his health, his amusements and his occupations, his diet and his desires, to be commanded and controlled in the smallest detail by apparatuses...”
CONSUMER Agamben, Giorgio. “What Is an Apparatus?” in What is an apparatus? and Other Essays. Stanford, California; Stanford University Press, 2009. pg 23.
Assemblage
61
Assemblage
Simon Rabyniuk Thesis Research 2018-19
Daniels Faculty of Architecture, Landscape and Design University of Toronto
supported by the Howarth-Wright Fellowship