The Mouse as Facilitator of Architectural Computation by Brooke Helgerson

THE MOUSE AS FACILITATOR OF ARCHITECTURAL COMPUTATION Histories of Digital Theory Final Paper Brooke Helgerson Washington University Fall 2012 Professor: Andrew Colopy

INTRODUCTION

The computer as we know it today has undergone a lifetime’s worth of changes and

advancements, most of which have occurred within our recent history. It was once a monstrous behemoth that took up entire rooms and was, for the layperson, shrouded in mystery. For those outside its developmental circles, its black and green screens controlled by scrambled lines of text did not play a large role in everyday life. The computer’s foreign programming language was a barrier to its ease use, and many wondered what it would ever be needed for.

Several input devices were eventually developed to address this problem, such as the chord-

key, light pen, and touchscreen.1 The one that rose above all of these in terms of ease of use and popularity was the mouse. It provided users with direct manipulation and immediate response of their movements on the computer screen. This development had effects on every professional field that utilizes computers. For many years it was not questioned as the best method of input. However, today it is much criticized in the world of architecture for being too Cartesian, and not representative enough of the way architects used to draw.2

Though computer drawing is different from analog sketching and drafting, it is this difference

that gives the mouse some of its most important qualities. By using a simple combination of usercontrolled motion and point-clicking, the mouse provided a simplified entry into the workings of the computer that has acted as a necessary transition into more gestural methods of working digitally. It is hard to imagine successful acceptance of a fully-formed digital drawing method as early as the 1960s; indeed, theorists like Paul Cantor and Steven Johnson have recognized that any new media needs time to acclimate to society before it can be exploited to its fullest potential.3 The simplified process of the mouse can also be seen to fulfill the requirements of moving use of the computer beyond computerization, which Sean Ahlquist and Achim Menges discuss as the conversion of analog processes into digital replicates, and into computation, or a recursive process of exploring ideas with the computer.4

Taking these ideas together, the mouse can be seen as more than a pointing device or a lesser

substitute for the pencil. The distinctive spatial interactions it sets provide a way of working that is an agent for computation and necessary step in the evolution of computer-based media. In order to understand the mouse’s position and future in the field of architecture, this paper will first look at its history of development, and how this relates to the interface. With this background, spatial analyses of the mouse’s capabilities will consider its place within the current trajectory of computation in architecture.

HISTORY Text Entry: Or Your Life without the Mouse

In order to truly appreciate the mouse, it is necessary

to imagine life before this evolution. Because it is so ingrained in the way we use computers today, this requires a leap of the imagination. It helps to look at an image of an early computer prototype, such as Tandy Radio Shack’s TRS 80 from 1977. [Figure 1] The lack of mouse, or even space for it on the desk the machine is sitting on, is evident. Imagine sitting here and performing the simplest of functions of your daily computer tasks. For example, how would you open a program? Or, in the context of an architectural office, how would you draw a line? Copy an object? Change a layer or color? On this machine, there is no visual way to understand these simple commands. Instead, users would have needed knowledge of a whole new programming language. Users had to know the desired outcome of each computer operation before it was entered into the machine. Additionally, in order to get the process to work correctly, each stream of text had to be entered 100% correctly; there was no margin for error, and no possibility to come across unexpected surprises.5

This transposition—from idea, to the elements needed to

realize it, to the programming language required to create the element, to the typing of the actual ‘sentence’—was a hindrance to users’ ability to use the computer with clarity [Figure 2] What may have started out as a seed of an idea that needed iteration to grow would get lost in translation, with the end result being less than what was imagined. For the most computer developers, this early disadvantage was not a large issue; after all, there were many other practical requirements being worked out. And because the computer’s earliest uses were in the military and engineering fields, not in design, this drawback did not get in the way too

Figure 1: The TRS 80 http://www.franklarosa.com/trs80/

much.6 However, some innovators were already investigating how computer technology could address these issues.

asf;lkgjfaghaskt::// bkjhvlcxkbasdf vkjafh oiuof-t8wurtwhb vaasfghsa kjlvhba kfbhfaklgv a asgs’g’:: fkjbhlaf afsg asdgfkajsh aty fkhfas g ;abhafs ksdf ckvhjfas wrtiyvicx hgafg akhafs gdsffdhjg afkgh sdkfgjh askjlhasdfas ‘salfgas gkf aaskdh asd :/asf g as laish alskh

asdfgh poiuyt xyz zxcvb qwerty ;’[]<>/ lkjhg asdfgh ;’[]<>/ poiuyt lkjhg qwerty xyz zxcvb

Figure 2: Diagram of an idea lost in translation, from conception to realization on an early computer. image by author

Key Development on the Coasts: MIT Media Lab and Sanford Augmented Human Intellect Research Center 1968: “If in your office, you as an intellectual worker, were supplied with a computer display, backed up by a computer that was alive for you all day and was instantly [sic.] responsive to every action you had, how much value could you derive from that?”7

1963: “We’re going to show you a man actually talking to computer, in a way far different than it’s ever been possible to do before…. He’s going to be talking graphically, with drawing, and the computer is going to understand his drawings.”8

The two statements above represent the work of Doug Engelbart and Ivan Sutherland,

respectively, towards developing a system of direct manipulation with the computer that would lead to the ‘mouse.’ Though the processes described seem standard to how computers work today, in the 1960s these abilities were brand new and just beginning to be figured out. Since then both the mouse, introduced in Engelbart’s 1968 demonstration, and the light pen, developed as part of Sutherland’s Sketchpad program, have had great effect on everyday use of computers. They have also greatly impacted how we design with computers—a topic that will be discussed further below, after further explanation of what these early systems were.

The Mouse and the Hyperlink

Douglas Engelbart, a researcher at the Stanford Research

Institute and eventual head of the Augmented Human Intellect Research Center, is known for being the ‘father of the interface.’9 The 1968 demonstration to engineers referenced above is where he introduced his NLS, or oN-Line System. This included the developments of the mouse, chord key, hyperlinks, and bitmapping that were some of the first steps towards creating a graphic (instead of text-based) interface for the computer. [Figure 3]

The idea of bitmapping, or assigning memory to each piece

(pixel) of the computer screen, is key to understanding how the

Figure 3: Engelbart in the 1968 demonstration. You can see him using the mouse, below, to manipulate the cursor, above. http://sloan.stanford.edu/ MouseSite/1968Demo.html#complete

mouse and hyperlink were able to allow for direct manipulation.10 [Figure 4] With this, each pixel could be accessed and expanded to reveal more stored information. The hyperlink associated with the pixel was the on-screen door to this information; the mouse asfgkhjlsag hlksdjahf alskxzckvjh savdk agskljahgkljxzchvkjlsa htwrklh dsbafdghas ghgpibhvn k;dfh afkldhvaskfhj sadlkfgj lkshga lkfgaskljfh erth ewiupghad fkbvdsaigqr[whtgdv kbhadfkjgaerghjdfkljh jkfhg lkaf kjdfg lkafdgh afslkgs klhasklgh klfh afklghaflghflkLetter Form ghdfkgh

was the input device that controlled which hyperlink was being Thesis

accessed.

Though Engelbart himself preferred the chord-key input

device to the mouse, its combination of keystrokes made it harder to learn. Several other factors, including cost and ease of use,

Figure 4: Diagram conceptualizing the idea of bitmapping image by author

led to the mouse winning out over other input devices being developed.11 The scale of motion of mouse operations (moving it around the worksurface as well as pointing and clicking) is also similar to human muscular capability, allowing users to manipulate it with efficiency.12

Sketchpad: Graphics into Architecture

Ivan Sutherland’s Sketchpad program, developed first

by him and subsequently by other researchers at the MIT Media Lab, was the precursor to the CAD softwares we use today. Its main innovation, beyond the creation of a light pen (similar to

Figure 5: The Sketchpad light pen and interface http://www.youtube.com/ watch?v=USyoT_Ha_bA&feature=relmfu

the pen styluses we use on tablets today, though heavier and more expensive13), was the ability to draw and manipulate shapes directly on the computer screen. [Figure 5] This was done by using the computer to connect points on the screen into lines, which could be joined to form shapes. The computer would translate the real path of the light pen and estimate the smoothest path, thus anticipating the ‘correct’ form that the user intended. These shapes could be dragged across the screen, scaled, and

Figure 6: Trackball 1952 http://www.billbuxton.com/inputTimeline.html (for figures 6-13)

its components edited; this enabled the ‘graphic communication’ discussed above by Stephen Coons in the Sketchpad Demo Video.14

Input Device Development and Distribution

Further advances in input technology maintained the

important developments started by Engelbart’s mouse and Sutherland’s Sketchpad: 1, that the mouse could work with

Figure 7: Light pen, 1957

hyperlinks to provide a graphical way to access the computer interface and 2, that line drawings could be made by connecting points determined by users through input devices. The details of how the hand interacted with the mouse to achieve these things are what changed the most.

Even in the early 1960s and 70s, there was a great

variety in the types of input being developed. [Figures 6-9] As the mouse became the device that was included with personal

Figure 8: Engelbart’s original mouse, 1964

computers,15 it became the most prominent among these. With the rise of personal computers, the form of the mouse developed significantly. This included a switch from wheels set at 90° to a 360° ball; the change in number of buttons, from two to three, to one, and then back to three with the addition of the scroll wheel; and an advancement from being attached by a cord to being wireless.16 [Figures 10-13] Today there is still a great variety of mice,

Figure 9: Touchscreen, 1965

the differences of which have been adapted to various uses across several fields.

INTERFACE AND REPRESENTATION Development of the Graphic User Interface

The computer interface developed significantly with the

introduction of the mouse. Instead of being based on text and

Figure 11: Multi-button mouse, 1985

programming language, the combination of mouse and hyperlink allowed pieces of the computer screen to be identified as objects. This concept was developed at XEROX’s Palo Alto Research Center (XEROX PARC) and resulted in what we now call the Graphical User Interface, or GUI.17 [Figure 14] This was imagined as a virtual replica of a typical work desk, and organized the vast information of the computer into familiar things like ‘files’ and ‘folders,’ accessed through different ‘windows’ on the ‘desktop.’

Figure 10: Single button mouse, 1989

[Figure 15] These were represented on the screen by icons that mimicked the look of actual files and folders, making it easy for users to navigate their stored data.18

The system of the GUI relies on the idea of skeumorphism,

or the practice of creating “an object or feature [that copies] the design of a similar artifact in another material.”19 The original inspirations for the on-screen icons impart their semiotic meaning and function to their counterparts on the computer. By shifting

Figure 12: Wireless mouse, 1991

away from a textual language to an object-oriented one, a layer of translation is absorbed by the interface. [Figure 16] Though this technically separates the user from the interface, the semiotic connection of working with the object provided a ‘tactile immediacy’20 that brought the user closer to understanding how to manipulate the items on the screen. Within a few clicks, they could access and manipulate any piece of information they had stored in the vast folders of their personal computer. [Figure 17]

Figure 13: 2-direction scrolling mouse, 1997

This type of interface worked so well because of the

mechanisms of the mouse: its ability to click makes it most suitable for distinct objects rather than streams of text. While it is true that information must be condensed for the computer to make the translation from text to a graphic icon, this reduction in complexity comes with the great benefit of making the mouse and computer usable by people with all types of backgrounds and levels of experience with computers. The computer term WYSIWYG, or ‘what you see is what you get,’ is derived from this one-for-one experience of the interface.21 Without this, computers would never have been so widespread today; instead they could

Figure 14: The XEROX Star, one of the first PC’s with the GUI http://design.osu.edu/carlson/history/ lesson16.html

still be limited to technology labs at large universities.

http://aliciabdesigns.blogspot.com/2008_09_01_archive.html

asdfgh poiuyt xyz zxcvb qwerty ;’[]<>/ lkjhg

http://www.webdesignerdepot.com/2009/03/operating-systeminterface-design-between-1981-2009/

Figure 15: Diagram of the GUI and its asdfgh ;’[]<>/ poiuytinspiration. lkjhg real-life qwerty xyz zxcvb diagrammatic elements added by author

Figure 16: Diagram of the easier communication between user and computer with the GUI image by author

Figure 17: Diagram of the ability to locate information within the structure of the GUI image by author

The Mouse and Communication: User Connection to the On-Screen World

With the mouse and direct manipulation came the idea of immediate feedback. When the user

moves the mouse, that motion is mirrored onscreen by the cursor. In this way, the user can project themself onto the screen and become and active part in the visual interface in front of them. Though every software assigns different functions to clicking, right clicking, etc., the principle of moving the mouse and reading that motion visually remains the same. Because of this, the cursor is a constant experience of using the computer. Whether or not we are actively using the mouse, the cursor reminds us of our position and allows us to adjust it with the mouse when necessary.

The mouse is how we communicate with the computer, and the cursor is how it speaks back

to us. This is the role of any input device: â&#x20AC;&#x153;to engage in dialogue with the machine.â&#x20AC;?22 The dialogue created through this process is quite different from how we usually communicate, by speaking.

Conversation through the mouse is more deliberate and one-sided than it is with people (after all, the computer in its current form can’t speak back to us or move the cursor in the same way we can). Though the technology for speech communication with computers has not been fully resolved,23 the preferencing of the hand-controlled mouse is something that occurred early on in input device development. This has set up a vantage point between the user and the screen that affects how we interact with the computer spatially, which has had great influence on how we design.

The Mouse and Drawing: Relationship to the Development of Perspective

This creation of vantage point, from ourselves to the computer, was a development enabled

by the mouse: “For the first time, a machine was imagined not as an attachment to our bodies, but as an environment, a space to be explored… [a] space worth living in.”24 The development of perspective technique offers a parallel to this phenomenon, albeit one outside of the digital world. The drawing tools this required, such as the straightedge or the triangle, perform similar roles to the mouse in this regard.

Both the constructed drawing and the constructed persona in the digital world are

intermediate tools that allow us to experience and understand spatial, or now, informational, data. [Figure 18] It is precisely the mouse’s abstraction from actual drawing gestures that allow for this new interpretation of space in the digital realm. Perspective drawings, by abstracting a three-dimensional space onto a two-dimensional surface, achieve a similar separation from the environment they represent. In so doing allow us to realize things we may not notice in the actual environment.

The connection of the mouse to perspective drawing can begin to form a counterargument

to the lamentation of computer drawing expressed by Michael Graves in his recent New York Times opinion article.25 The mouse and cursor, through their relationships with the hand and eye, fulfill the “…interaction of our minds, eyes, and hands” that Graves requires of drawing.26 Like the perspective drawing (which must be an acceptable form to Graves, at least when hand-drawn), the mouse engages the creative process by constructing a spatial relationship with a representational environment. It can perform the functions of the ‘referential’ and ‘preparatory’27 sketches by giving users access to different programs and toolbars, layers of options which must be sorted through and understood by the user in a similar way to understanding the options for building in a hand-drawn sketch.

Z y

x’

y’

http://longstreet.typepad.com/thesciencebookstore/2009/01/looking-straight. html

x x

Figure 18: Diagram of relationship between perspective drawing technique and the mouse’s spatial structure image by author

SPATIAL ANALYSIS Mechanisms of the Mouse

Through their innovative systems, Johnson and Sutherland “…endowed the digital computer

with space;”28 the mouse itself opened up a world of spatial relationships between the user and their virtual world. But what now? What, exactly, does the mouse inscribe on the architectural process?

First, the mouse a touch-based input (as opposed to speech or other communications)

that operates in two dimensions. It is single-touch (as opposed to multi-touch, which would read information from more than one finger at a time).29 It translates pointing and clicking, double clicking, or scrolling into sequential operations on the computer screen.

Because of these preconditions, the mouse is better able to draw vectors than curves. For

example, it is no problem to draw the points of a complex polygon with a mouse, but signing your name accurately may pose a problem.30 This deficiency has not posed huge barriers to the use of the mouse, even in design. Several programs that work in complex curvatures still create them using a system of weighted points, or NURBS (Non-uniform Rational B-Splines).i However, it is important to recognize that the mouse does some things better than others.

How the Mouse Operates in Common Design Softwares AutoCAD and Illustrator

The mouse works quite well with AutoCAD, which is a vector program that connects lines

between points drawn by the user. The application’s addition of things like orthogonal locking and object snaps to the cursor make the mouse a quite accurate tool for drawing construction documents. The mouse’s ability to manipulate objects is also expanded by a combination of hot keys, or short typed key combinations that allow the cursor to immediately take on different types of manipulations of the drawing, such as scale or rotate. Even with this though, the mouse is the primary tool for drawing the actual points. One drawback comes when clicking and hot key combinations are required. For example, when the user needs to zoom, they must use the scroll wheel and a key such as control or command, thus taking up both hands and reducing the ability to move on to the next command. Here, the single-touch and linear input of the mouse become limitations.

The mouse works quite similarly in Adobe Illustrator: lines are manipulated by points, and are

suited to the linear structure of the mouse. There is also a very useful set of quick keys codes that greatly expand the mouse’s efficiency when changing from tool to tool via the cursor. However, this program’s ability to draw continuous lines (the application applies the points automatically) would benefit from a more natural range of motion. Because of how the mouse is positioned in the hand, it is actually quite difficult to replicate the form we are used to when holding an analog drawing utensil; thus, smooth lines come out rather choppily.

In order to draw these arguments to the unique qualities the mouse allows for, 3D modeling

programs and Photoshop will only be referred to here briefly. 3D programs like Revit and Rhino work similarly to CAD, but have the added requirement of orbiting, which sometimes becomes cumbersome, similar to zooming. Photoshop is even more stroke and brush based than Illustrator, and so relies even more on a way to replicate drawing. Additionally, it should be said that all of these specifics are chosen at the discretion of the author; there are countless many more benefits and drawbacks to the use of the mouse as an input device that depend on the way of working of the user, which is a consideration beyond the use of the mouse alone.

The World of the Toolbar

While the above is an assessment of how well the mouse can draw in terms of the options

provided by software, one of its most important benefits comes from its ability to allow the user to

change identity, both between applications and within them. This capability is accessed from each program’s toolbar and is manifested in the cursor, the user’s virtual representative.

The tools (another skeumorphic analogy of the GUI) provided in each program allow the

mouse to perform completely different functions, simply depending on which one is selected. It can be a brush, a scissor, a scaling tool, or a color-changing tool, all with one click. The succession of what the mouse can be, depending on the program, is endless, and has no equivalent in analog drawing. This allows the mouse to perform operations not possible by drawing or painting alone, enabling it to question image properties and discover new relationships between them. With things like filter galleries, modify or build toolbars, or brush types, the mouse can turn the computer into a speculative, not just a representational, tool. In this way, it can start to grasp the “…new potentials in unexpected mixtures of the digital and analog, the real and the virtual, or the everyday and the fantastic” that Stan Allen advocates for in “The Digital Complex.”31 The mouse allows for this connection not in spite of, but because of its one-step removal from the literal methods of drawing.

Criticisms

For most computer users, who either use it recreationally for the internet or deal in

documentation and spreadsheets, the mouse is a perfectly capable tool. But Nicholas Negroponte, through an analogy made by Neil Gershenfeld at the MIT Media Lab, points out that the mouse is no cello bow.32 In other words, it does not have the sensitivity that may be required for an experienced user to truly draw out the capabilities of the instrument—in this case, the computer.

This criticism adds to that of Michael Graves, discussed above. He argues that computer-

based design, specifically parametric design, is not enough to match the spontaneous creativity of a hand sketch.33 Eugenia Ellis brings up further concern for the computers ability to provide an experience of the real world in an article for the Journal of Architectural Education. Here, she argues that the use of the computer filters our experience through technology rather than actual experiences, and makes the point that space can only be understood from being in it, rather than perceiving it through the lens of a computer program.34 In this understanding of the computer in design, the mouse and its manipulations would only provide a paltry estimation of the space or graphic being developed. This is an extreme view, however, and is countered by the study of mirror neurons conducted by David Freedberg and Vittorio Gallese.35 Their study shows that a physical response actually can be stimulated by viewing a work of art, even if it has no overt gestures of

movement.36 This suggests that our interactions with space via the mouse and computers are not so empty as Ellis claims. While this doesn’t erase the need for an input device that is more sensitive to our capabilities as designers, it does provide support for the contention that the mouse allows us to expand on drawing practices and enter into the realm of computation.

The mouse has also provided another important function, of acting as a stepping-stone

in technology to prepare us for new forms of media computers and the input device can bring. Similarly to the familiar icons of the GUI, the simplified motions of the mouse would have seemed unthreatening to the general public. This seems a silly thing to say, but the phenomenon of the public doubting emerging technologies is widespread. Developments as diverse as electricity and cinema have been considered with suspicion.37 Walter Benjamin’s fear of the loss of ‘aura,’ or unique qualities of an object that are rooted in their time and place of production, in the face of industrialization, is another example of this concern.38 Paul Cantor, a literary and media critic, makes sense of this fear by discussing how all new media undergoes a process of simplified novelty before it is able to realize or express its true power (he points out that even Shakespeare’s plays and Dickens’ serial novels were once considered unconventional).39

The fact that our current computer input methods are being criticized may indicate that the

input device as a type of media is ready to take on new form. This does not mean that the mouse in its current form has been elementary, however. The methods it offers still have widespread use and functionality, in addition to it serving as an important component of the evolution of digital technology.

Speculation

The fact that the mouse preferences the vector may be attributable simply to technology’s

ability to imagine what it could do at the time it was developed. In the 1960s, the main focus of computer technicians was to translate input data into output. Our computer capabilities are no longer so binary, and involve the interpretation of things like images, algorithms, and complex search systems. Perhaps these advances have prepared input technology to expand beyond the mouse.

Several prominent authors and researchers in the digital field have discussed ways of

creating a symbiotic relationship between man and computer.40 This process may involve things like developing speech recognition software, so that the computer can understand our nuanced commands while allowing us to work hands-free.41 It could even become something where the

computer recognizes us visually, by recognizing where our eyes are directed.42

This new development may also still relate to our sense of

touch. However, the one or two click system of the mouse may be changed into one that reads multiple nodes of touch.43 This is already beginning to happen, most readily in the trackpad found on Apple laptops. These sensors can read the difference between one and four finger swipes, and also allow for the legibility of pinching and pulling motions to scale and rotate windows and objects. There are also a few newer solutions such as Sixth Sense, which does away with the ‘input device’ altogether and places the

Figure 19: Pranav Mistry demonstrating his Sixth Sense technology, being developed at MIT http://latd.com/2009/11/29/sixthsensein-the-everyday-fewer-devices-betterconnection/

sensors directly on the users own fingertips.44 [Figure 19] With this technology, we may be getting closer to a real life push-pull ability that emulates technology fictionalized in our pop culture. [Figure 20]

CONCLUSION

When speculating about the mouse’s future possibilities, it

is easy to adopt the mindset that its current form is a dumb tool with a low degree of precision. However, its mechanisms can be read as an important product of the era in which it was developed. The developments of Engelbart’s NLS and Sutherland’s Sketchpad were both instrumental to the specific trajectory of the mouse. The GUI that resulted from the hyperlink and bitmap directed the mouse’s future development to be object and image-oriented. This has had great effect on the way we use the mouse in architectural design. Though its methods don’t emulate analog drawing, they are set up to provide a system of computation that allows users to visualize themselves in the virtual world. Through the mouse and cursor, they can take on new identities from the various toolbars programmed into applications. This relationship

Figure 20: The haptic manipulations seen in pop-culture films such as Iron Man may actually suggest where input devices could be headed http://www.apartmenttherapy.com/ movie-technology-wed-like-in-ourhome-office-172356

may not have developed if the methods of input had not been abstracted from the complexity of gesture to the linearity of the mouse. Even if input devices trend towards gesture in the future, the presence of the mouse has been significant to the way we think about our interactions with the computer, and will no doubt continue to exert influence as input device technology proliferates.

Bill Buxton, “Some Milestones in Computer Input Devices: An Informal Timeline,” Microsoft Research Center, entry posted April 4, 2012, http://www.billbuxton.com/inputTimeline.html Accessed October 24, 2012. 2 Michael Graves, “Architecture and the Lost Art of Drawing,” New York Times Sept. 1, 2012. 3 Paul Cantor, “Commerce and Culture Lecture 10: Culture as Pop Culture,” Presented by the Ludwig von Mises Institute, July 24-‐29, 2006. Accessed at <http://www.mises.org/media.aspx?action=category&ID=91> 17 December 2012; Steven Johnson, Everything Bad is Good for You, (New York: Riverhead Books, 2005) and Interface Culture: How New Technology Transforms the Way We Create & Communicate, (New York: Basic Books 1997). 4 Sean Ahlquist and Achim Menges, “Introduction: Computational Design Thinking,” Computational Design Thinking (2011), 10-‐11. 5 Ivan Sutherland, “Part 3: Historical Perspective: “Computer Sketchpad,”” Sketchpad demo video, MIT Media Lab 1963, http://www.youtube.com/watch?v=USyoT_Ha_bA&feature=relmfu Accessed October 24, 2012. 6 Kiel Moe, “Automation Takes Command: The Nonstandard, Unautomatic History of Standardization and Automation in Architecture,” Fabricating Architecture, (New York: Princeton Architectural Press, 2010), 159-‐165. 7 Doug Engelbart, Demonstration of NLS, given by the Stanford Augmented Human Intellect Research Center to the Fall Joint Computer Conference in San Francisco, December 9, 1968, http://sloan.stanford.edu/mousesite/1968Demo.html Accessed October 24, 2012. 8 Stephen Coons in Sutherland. 9 Steven Johnson, Interface Culture: How New Technology Transforms the Way We Create & Communicate, (New York: Basic Books 1997), 14. 10 ibid., 20. 11 “Revolution: The First 2,000 Years of Computing: Input & Output,” Exhibition, Computer History Museum, http://www.computerhistory.org/revolution/topics#exhibition Accessed October 24, 2012. 12 Stuart K. Card, Jock D. Macinlay, and George G. Robertson, “A Morphological Analysis of the Design Space of Input Devices,” ACM Transactions on Information Systems 9 no 2 (April 1991): 100. 13 Negroponte, Nicholas. Being Digital. New York: Knopf Publishers 1995, 131. 14 Sutherland. 15 Revolution Exhibit. 16 Buxton, “Some Milestones”. 17 “The GUI and the Personal Computer,” in A Critical History of Computer Graphics and Animation, The Ohio State University online resource, http://design.osu.edu/carlson/history/lessons.html Accessed October 24, 2012. 18 ibid. 19 “Skeumorph,” Oxford English Dictionary Online. Accessed 17 December 2012. 20 Johnson, 21. 21 “The GUI and the Personal Computer.” 22 Card et. al., 101. 23 Negroponte and J.C.R. Licklider, “Man-‐Computer Symbiosis.” IRE Transactions on Human Factors in Electronics v. HFE-‐1, 1960. 24 Johnson, 24-‐25. 2525

Michael Graves, “Architecture and the Lost Art of Drawing,” New York Times, Sept. 1, 2012. ibid. 27 ibid. 28 Johnson, 47. 29 Wayne, Westerman, Hand Tracking, Finger Identificaion, and Chordic Manipulation on a Multi-‐Touch Surface, Dissertation for University of Delaware, 1999, xxvi. 30 Bill Buxton, “Chapter 1: An Introduction to Human Input to Computers,” Human Input to Computer Systems: Theories, Techniques, and Technology, 2011, Unpublished personal work accessed online: http://www.billbuxton.com/inputManuscript.html, 7. 31 Stan Allen, “The Digital Complex,” Log 5: Observations on Architecture and the Contemporary City (Spring/Summer 2005): 94. 32 Nicholas Negroponte, Being Digital, (New York: Knopf Publishers 1995), 130. 33 Graves. 34 Eugenia Victoria Ellis, “Ceci Tuera Cela: Education of the Architect in Hyperspace,” Journal of Architectural Education v. 51 (Sept. 1997): 41-‐42. 35 David Freedberg and Vittorio Gallese, “Motion, Emotion, and Empathy in Esthetic Experience,” i n Aesthetic Theory, ed. by Mark Foster Gage, (New York: W.W. Norton and Company, 2011). 36 ibid., 315. 26

http://www.billbuxton.com/inputManuscript.html, 7. 31 Stan Allen, “The Digital Complex,” Log 5: Observations on Architecture and the Contemporary City (Spring/Summer 2005): 94. 32 Nicholas Negroponte, Being Digital, (New York: Knopf Publishers 1995), 130. 33 Graves. 34 Eugenia Victoria Ellis, “Ceci Tuera Cela: Education of the Architect in Hyperspace,” Journal of Architectural Education v. 51 (Sept. 1997): 41-‐42. 35 David Freedberg and Vittorio Gallese, “Motion, Emotion, and Empathy in Esthetic Experience,” i n Aesthetic Theory, ed. by Mark Foster Gage, (New York: W.W. Norton and Company, 2011). 36 i bid., 3 15. 37 Linda Simon, Dark Light: Electricity and Anxiety from the Telegraph to the X-‐Ray, (New York: Harcourt Inc., 2004); and Dave Kenney, Twin Cities Picture Show: A Century of Moviegoing, (St. Paul: Minnesota Historical Society Press, 2007). 38 Walter Benjamin, “The Work of Art in the Age of Its Technological Reproducibility,” in The Work of Art in the Age of Its Technological Reproducibility and Other Writings on Media, (Cambridge, MA: Harvard University Press 2008), 21. 39 Cantor, Paul. “Commerce and Culture Lecture 10: Culture as Pop Culture,” Presented by the Ludwig von Mises Institute, July 24-‐29, 2006. Accessed at <http://www.mises.org/media.aspx?action=category&ID=91> 17 December 2012. 40 Negroponte and J.C.R. Licklider, “Man-‐Computer Symbiosis,” IRE Transactions on Human Factors in Electronics v. HFE-‐1:4-‐11, 1960. 41 ibid., 137-‐148. 42 ibid., 135. 43 Westerman. 44 Pranav Mistry, “Sixth Sense,” MIT Media Lab online resource, http://www.pranavmistry.com/projects/sixthsense/ Accessed October 24, 2012.

Authors note: Though this is true, there may be a discussion raised here about whether the NURBS method of creating curves remains based on points precisely because of the mouse’s preference for them. This is a consideration that would require further research.

BIBLIOGRAPHY A Critical History of Computer Graphics and Animation. The Ohio State University online resource. http://design.osu.edu/carlson/history/lessons.html Accessed October 24, 2012. Ahlquist, Sean and Achim Menges. “Introduction: Computational Design Thinking.” Computational Design Thinking (2011): 10-29. Allen, Stan. “The Digital Complex.” Log 5: Observations on Architecture and the Contemporary City (Spring/Summer 2005): 93-99. Benjamin, Walter. “The Work of Art in the Age of Its Technological Reproducibility.” in The Work of Art in the Age of Its Technological Reproducibility and Other Writings on Media. Cambridge, MA: Harvard University Press 2008. Buxton, Bill. “Some Milestones in Computer Input Devices: An Informal Timeline.” Microsoft Research Center, entry posted April 4, 2012, http://www.billbuxton.com/inputTimeline.html Accessed October 24, 2012. Buxton, Bill. “Chapter 1: An Introduction to Human Input to Computers.” Human Input to Computer Systems: Theories, Techniques, and Technology. 2011. Unpublished personal work accessed online: http://www.billbuxton.com/inputManuscript.html. Card, Stuart K., Jock D. Macinlay, and Goerge G. Robertson. “A Morphological Analysis of the Design Space of Input Devices.” ACM Transactions on Information Systems 9 no 2 (April 1991): 99-120. Graves, Michael. “Architecture and the Lost Art of Drawing.” New York Times Sept. 1, 2012. Ellis, Eugenia Victoria. “Ceci Tuera Cela: Education of the Architect in Hyperspace.” Journal of Architectural Education v. 51 (Sept. 1997): 37-45. Engelbart, Doug. Demonstration of NLS, given by the Stanford Augmented Human Intellect Research Center to the Fall Joint Computer Conference in San Francisco, December 9, 1968. http://sloan.stanford.edu/mousesite/1968Demo.html Accessed October 24, 2012. Freedberg, David and Vittorio Gallese. “Motion, Emotion, and Empathy in Esthetic Experience.” in Aesthetic Theory, ed. Mark Foster Gage, 309-323. New York: W.W. Norton and Company, 2011. Johnson, Steven. Interface Culture: How New Technology Transforms the Way We Create & Communicate. New York: Basic Books 1997. Licklider, J.C.R. “Man-Computer Symbiosis.” IRE Transactions on Human Factors in Electronics v. HFE-1:4-11. 1960. Mackinlay, Jock, Stuart K. Card, and Goerge G. Roberston. “A Semantic Analysis of the Design Space of Input Devices.” Human-Computer Interaction v. 5, (1990): 145190.

Mistry, Pranav. “Sixth Sense.” MIT Media Lab online resource. http://www.pranavmistry.com/projects/sixthsense/ Accessed October 24, 2012. Moe, Kiel. “Automation Takes Command: The Nonstandard, Unautomatic History of Standardization and Automation in Architecture.” in Fabricating Architecture. New York: Princeton Architectural Press, 2010, 152-67. Negroponte, Nicholas. Being Digital. New York: Knopf Publishers, 1995. “Revolution: The First 2,000 Years of Computing: Input & Output,” Exhibition, Computer History Museum. http://www.computerhistory.org/revolution/topics#exhibition Accessed October 24, 2012. Rheingold, Howard. Tools for Thought: The History and Future of Mind-Expanding Technology. New York: Simon & Schuster 1985. Steenson, Molly Wright. “Computing, Computer-Aided Design, Media: From ComputerAided Design to the Foundations of Interactivity.” Architecture School: Three Centuries of Educating Architects in North America. Ed. Joan Ockman. Cambridge, MA: MIT Press 2012. Sutherland, Ivan. “Part 3: Historical Perspective: ‘Computer Sketchpad.’” Sketchpad demo video. MIT Media Lab 1963. http://www.youtube.com/watch?v=USyoT_Ha_bA&feature=relmfu Accessed October 24, 2012. Vora, P.R. “Hypertext and Hypermedia.” Denver, Colorado: Taylor & Francis Group, LLC, 2006. Westerman, Wayne. Hand Tracking, Finger Identificaion, and Chordic Manipulation on a Multi-Touch Surface. Dissertation for University of Delaware. 1999.