r e a l i z i n g
r e a l i z i n g
In memory of Dr. Randolph “Randy” Pausch 10. 23. 1960 – 07. 25. 2008
Thank you for changing my life.
Copyright © 2011 by Seung Chan Lim Some rights reserved.
To view a copy of this license, please visit : creativecommons.org/licenses/by-nc-nd/3.0/ Or send a letter to : Creative Commons 444 Castro Street, Suite 900 Mountain View, C A 94041 USA To contact the author, please visit: twitter.com/seungchan/
time f r o m
o u g h a r t a l o n e are we able to emerge from ourselves, to know what another person sees of the universe which is not the same as our own and of which, without art, the landscapes would remain as unknown to us as those that may exist in the moon. Thanks to art, instead of seeing one world only, our own, we see that world multiply itself and we have at our disposal as many as there are original artists, worlds more different one from the other than those which revolve in infinite space, worlds which, centuries after the extinction of the fire from which their light first emanated, send us still each one its special radiance. Marcel Proust / n o v e l i s t ,
cr itic, a nd essayist
It goes without saying that this book is not a work of an individual, but of a collective: a collective that has taught me the value of love and trust. As you will no doubt witness hereafter, the insights shared across these pages result from being around brilliant individuals rather than being in solitude. I would like to take this space to express my sincere gratitude to the individuals who have influenced me the most. I would like to first of all thank my parents. After spending many years trying to make the best choices in life, I have come to realize that the one thing I didn’t get to choose has been the best. 어머니, 아버지 진심으로 감사드립니다. 낳아주신 은혜 평생 잊지 않을게요. I haven’t the slightest clue how I will repay my debt to my three thesis advisors, David Gersten, Paul Pangaro, and Thomas Ockerse. Needless to say, the different perspectives you all have shared with me on topics ranging from semiotics to phenomenology to conversation theory have been invaluable, but above all, I cannot thank you all enough for your trust, patience, and encouragement. You all have been incredibly generous with your time and patience. And for that, I am infinitely grateful. I would also like to extend my deep appreciation to An-Lon Chen, Anson Ann, David Watson, Jeff Wong, and Joonkoo Park for accompanying me in this peculiar journey that is the m fa thesis writing. Your feedback has opened up new avenues of possibilities for future research. I would like to thank An-Lon in particular for her extensive help making the final edits. You all are living proof that empathic conversations can indeed take place online, even when the venue is f a c e b o o k or p o s t e r o u s — still not sure about t w i t t e r , though. ;)
Across both the r h o d e and b r o w n
isl a nd school of de sign
u n i v e r s i t y,
I have been fortunate
enough to have taken classes taught by those who exemplify the very act of listening in their teaching. In particular, I would like to thank Anne West, Chris Rose, Connie Crawford, Doug Scott, George Gordon, Peter Prip, and Tucker Houlihan for being so generous with your time responding to my seemingly endless chain of questions. A very special thanks goes out to my fellow students for teaching me the meaning of courage. In particular, my heart goes out to the 2008 Freshmen class section 20. It was an unbelievable act of luck to have been with you guys on the first day of graduate school. To this day I cannot forget the valuable lessons I was able to glean from merely being in your presence. I would also like to thank Yuki Kawae for taking interest in my thesis and inspiring me with your own stories. May the conversations live on in the Wood Shop. I would like to acknowledge the influence of Peter Lucas who has essentially shaped my foundational knowledge of Computer Science and its relationship to the human mind, Ju Hyun Kang who has given me a once-in-a-life time opportunity to experience what an act of empathy actually entails, and Randy Pausch without whom I would have never understood the value of Computer Science. Finally, it is very difficult to imagine whether I would have had the energy to pursue three years of relentlessness had it not been for Yong Joo Kim who has stood by me with her sense of humor and positive attitude. Thank you so much. You inspire me like nobody I have ever known.
Eames Century Modern by h o u s e i n d u s t r i e s Rockwell Std by m o n o t y p e 산돌북 by s a n d o l l Bell Centennial Std by m at t h e w
c o m m u n i c at i o n
Gotham Condensed by t o b i a s f r e r e - j o n e s Printed on x e r o x , 17.7lb 84 Brightness, with h p ® Color LaserJet cp2025dn Bound by Hope Bindery & Box Co. Designed by Seung Chan Lim Written by Seung Chan Lim Edited by An-Lon Chen List of photographs used under the Creative Commons Attribute 3.0 Unported License: p. 20-21 u n t i t l e d by m i l e s g e h m • milesgehm on f l i c k r • flickr.com/photos/milesgehm/ 308444650/
p. 28- 29
s e n e c a’ s c o m p u t e r l a b
by j o e y d e v i l l a • Joey Devilla on f l i c k r • flickr.com/photos/ accordionguy/4572741782/
p. 46- 47
t h e g a r a g e by barto on f l i c k r • flickr.com/ photos/barto/28135419/
p. 312-313 s p i n n ’ g l a s s by b r i a n h i l l e g a s • Brian Hillegas on f l i c k r • flickr.com/photos/ seatbelt67/3878890767
p. 376- 377 c r u s h e d by m i k e m c c u n e • mccun934 on f l i c k r • flickr.com/photos/mccun934/ 2899155411/
After a nine-year career in Computer Science and Interaction Design, I have spent the last three years immersed in dialogue with physical materials — including the human body. Through the practice of acting, dancing, writing, and working with paper, clay, glass, wood, metal, plastic, plaster, light and type, I realized that making things with physical materials is analogous to engaging in an empathic conversation with another person. Based on this experience, I have distilled and developed a list of five necessary qualities — a set of shared metaphors, and a sense of trust, honesty, integrity, and dignity — that our interaction with the computer must afford, before it can facilitate an empathic conversation between software computation and the human body.
Somewhere around every seven years make a significant, if not complete, shift in your field. Thus, I shifted from numerical analysis, to hardware, to software, and so on, periodically, because you tend to use up your ideas. When you go to a new field, you have to start over as a baby. You are no longer the big muckety muck and you can start back there and you can start planting those acorns which will become the giant oaks. Richard Hamming / m a t h e m a t i c i a n
As an American mathematician whose work had many implications for computer science and telecommunications, Richard Hammingâ€™s contributions include the Hamming code (which makes use of a Hamming matrix), the Hamming window, Hamming numbers, Sphere-packing (or hamming bound) and the Hamming distance. He was a founder and president of the a s s o c i at i o n f o r c o m p u t i n g m a c h i n e r y . His philosophy on scientific computing appears as the preface to his 1962 book on numerical methods: The purpose of computing is insight, not numbers. source: wikipedia
Acknowledgements p. 4 Abstract p. 9 Preface Computer Trouble p. 21 Glossary p. 42
Realizing Empathy p. 45 Interview
Empathy and Risk-Taking p. 110 Stories
When Words Become Metaphors p. 177 Listening to Honesty p. 221 Respecting Integrity p. 285
A New Day in the Acting Studio
Projects The Personal Computer Revolution p. 38 Physics as Freedom p. 100
Computation as Risk, Computer as Facilitator
Type in Space p. 208 Listening to Honesty p. 266
What is That on Your Desktop?
To Tear p. 290 Learning from Differences p. 316 Principles of Empathic Conversation p. 380 Human Computer Interaction p. 382 Model of Empathic Conversation p. 410 Postscript A Different Way Forward p. 393
c o m p u t e r
I attended as an undergraduate 1
I graduated from Carnegie Mellon’s School of Computer Science in 1999
was filled with an amazing roster of wellknown Computer Science professors. Whenever I mentioned their names to students of Computer Science at other universities, they’d remark, “Wow! You’re
studying under that professor ? ! That’s amazing! Did you know that he invented the algorithm for . . .”
This is not to
Well, unfortunately, what I learned from these professors was actually quite minimal. As a matter of fact, I was more confused about Computer Science after having taken their classes.
point fingers, but to admit to the fact that learning Computer Science through lectures and mathematical formulas was insufficient for me. Despite the rigorous mathematical training I had received in high school, I had the hardest time making sense of the concepts discussed in Computer Science, and learning how to program. As a result, I felt like a giant ball of failure, utterly confused and miserable. But during the second semester of my junior year, I took a class called computer architecture, which I followed up with another class called operating system design and implementation. Little did I know that these two classes
would radically change my understanding of Computer Science.
Preface | computer trouble
In Computer Architecture,
we discussed what was inside a modern day computer. We learned how and why different hardware parts were designed, and also how they inter-operated with one another. When it came to assignments, we wrote programs that demonstrated our understanding of the behaviors and constraints of these hardware parts.
In Operating System Design and Implementation,
we discussed the design principles behind a modern day Operating System, and also learned how it interacts with available physical resources such as memory, processor, and hard disk storage units. To demonstrate our understanding of these ideas, we spent the semester programming an Operating System from the ground up.
So why were these classes any different than the others? Well, first of all, instead of focusing on the abstract mathematical concepts, these classes focused on the physical reality. And while I was trying to understand the behaviors and constraints of the available physical resources, I had a profound moment of realization; the abstract programming concepts I was struggling with had a concrete physical base! As much as all the previous professors wanted to make Computer Science seem like a purely abstract discipline, I learned that many of the programming concepts were the way they were because of the physical constraints they had to deal with. All of a sudden, the abstract concepts, which had given me such difficulty in the past, started to make a lot of sense! I could not help but wonder why I had not realized this before. Had I learned about this first, would I have had as much trouble understanding the abstract ideasâ€Š?
Why were these the last classes to take and not the first?
Preface | computer trouble
It is very easy to brush this anecdote off
as yet another instance of a young grasshopper who had to go through long hard training before she could snatch the pebbles. Unfortunately, I do not think that this is the case here. What took me three years to realize was that computers are physical. What they actually do is provide physical constraints such as memories, processors, and external storage units for programmers to work with. So it follows that the abstract paradigms found in programming languages can be better understood given a more intimate knowledge of hardware. Could this not have been addressed had the curriculum been structured differently? But I am not here to push for an overhaul in the undergraduate Computer Science curriculum. If this were merely an issue with university Computer Science curricula, I would not be so concerned. What troubles me is that the experience is far worse
for those who do not study Computer Science.
How many people who have not studied Computer Science know why software crashes, why we run out of hard drive space all of a sudden, why the computer gets slower and slower, or why we have to buy software to read a document? None that I know. Yet they fall victim to these events day in and day out. What happens when they do?
They feel frustrated, and powerless.
Why? Because they have no visibility into what is actually going on. They do not know how to respond, let alone prevent it from happening in the future. As a result, they end up blaming themselves for doing something wrong, or proceeding to purchase the latest and greatest in computer hardware, hoping the problem will somehow go away, which, of course, it never does.
Preface | computer trouble
The profound effect of such opacity becomes even more alarming once you realize that this is not an issue uniquely tied to hardware. Have you ever asked a designer to explain how you can convert your colored photos into black and white? If you ever do, they will inevitably start walking you through a series of actions that involve opening an application such as Photoshop, then moving your mouse to find and locate a button or a menu item to click on. But what have they explained about the actual process of turning a colored photo into black and white? If a dobe decided to move the menu item to a different location, and rename their labels, how useful would this knowledge be in that new context? While the designer might think she knows how something is done, what she actually knows is how to go through the motions without the slightest clue as to what they mean. This is essentially what it would look like had Henry Fordâ€™s assembly line been taken out of factories, and re-stationed at our homes.
Day by day, people everywhere are becoming enslaved to the whim of their favorite software applications, as we willingly slide our mouse, and click the on-screen buttons without ever questioning
the loss of autonomy we endorse along the way.
Preface | computer trouble
The fact of the matter is, most people have absolutely no idea what goes on inside the computer, whether it is hardware or software-related. One may be led to
believe that this is because computers are complex, but it is not.
It is because the general trend — in both Computer Science education and hightech product design — has been to hide instead of to reveal, to make abstract instead of to make concrete. It is by design. As a matter of fact, this trend can be found all around the product design industry. In shop class as soulcraft, researcher Matthew Crawford illustrates this phenomenon by citing the fact that Vacuum cleaners no longer come with detailed schematics, and that some of the new Mercedes-Benz automobiles lack the dipstick, making it difficult for the owners to maintain the oil level themselves.
In an era where people demand to know where their food comes from, how their tax dollars are being spent, or how much power their appliances use, it is ironic that we can stay calm about an invention that refuses to reveal its inner-workings while being so deeply integrated into our lives. Why don’t we demand that we have a way of knowing what is actually going on? That we have a say in whether our software should allocate so much memory as to risk not being responsive to our actions? That we be able to see how Photoshop converts colored photos to black and white, so that I can later do it with “my own bare hands?” Is this an issue of education and awareness? Perhaps. But framing it as such presumes that the solution is to teach everyone the ins and outs of the computer hardware. No. Unfortunately, the problem is
What if this is an abusive relationship that we are engaged in? The kind where we know that we are being abused, but we feel that we have no choice but to stay in?
The biggest question asked about an abusive relationship is why the abused don’t just leave the abusers. The reason why they don’t is not because they are unaware of the abusive nature of the relationship, or because they don’t know how to run away. The reason is because they are afraid of the consequences. The promise of even greater violence, for instance, is ever so imminent, that leaving is not considered an option. What if — unbeknownst to ourselves — we put up with the current incarnation of the personal computer because we think that we have no choice but to accept it as it is? How many of us feel confident that we can prevent software from crashing? Edit a digital photograph without purchasing a software application? Fundamentally influence the computer’s decision making? How many of us
can choose not to use the computer?
Even the more advanced users of the highly customizable Linux Operating System can feel physically unstable, when they encounter problems compiling and installing new software that leave the system in an unknown state. Online forums are filled with the outcries of such individuals trying desperately to restore their sense of balance.
All these add up to an undeniable feeling that there is something deeply troubling about the current state of personal computing. It is not simply the high degree of abstraction found in programming languages, the opacity afforded by software applications, the complexity of maintaining customized personal computers, or even the oftendiscussed virtualities of our online identities. It seems to be all of the above, and much much more. What are we
Preface | computer trouble
What I would like for us to have is a philosophy of Computing. One that can guide us to design the next generation of personal computers with a deep sense of ethics. Not the kind of ethics that preaches a moral and virtuous life, but one that embraces the imperfections of humanity, and the vulnerabilities that stem from our inter-dependencies. The kind of ethics that — as the philosopher Emmanuel Levinas 2 might say — embody the wisdom of love rather than the love of wisdom. One whose goal is not to humanize technology or to be human-centric, but to simply facilitate a conversation between the space of computation and the space of the human body.
Because we need it — now,
Emmanuel Levinas was a Lithuanian-born French Jewish philosopher and Talmudic commentator. His work is based on the ethics of the Other or, in Levinas’ terms, on “ethics as first philosophy”. For Levinas, the Other is not knowable and cannot be made into an object of the self, as is done by traditional metaphysics (which Levinas called “ontology”). Levinas prefers to think of philosophy as the “wisdom of love” rather than the love of wisdom (the literal Greek meaning of the word “philosophy”). In his view, responsibility precedes any “objective searching after truth”.
more than ever —
to restore a sense of dignity to our relationship with technology.
t he per son a l compu ter r e vo lu tion (20 09) is a poster visualizing the frustrations we feel when interacting with our personal computers. By juxtaposing a pixelated symbol of a lemon, made entirely of images depicting the various ways in which we interact with personal computers, with a famous quote by a renowned Computer Scientist, Alan Kay, the work reminds us that what we have now can’t possibly be the final product of a revolution.
seung chan lim
di g i ta l p r i n t
25˝ x 36˝
The Personal Computer Revolution Has Yet to Happen.
S L I M RI SD 2009
Our societyâ€™s growing reliance on computer systems that were initially intended to help people make analyses and decisions, but which have long since both surpassed the understanding of their users and become indispensable to them, is a very serious development. Joseph Weizenbaum / a u t h o r ,
As a German-American author and professor emeritus of computer science at m i t , Joseph Weizenbaum published a comparatively simple program called e l i z a , named after the ingenue in George Bernard Shawâ€™s Pygmalion, which performed natural language processing. Driven by a script named d o c t o r , it was capable of engaging humans in a conversation which bore a striking resemblance to one with an empathic psychologist. Weizenbaum modeled its conversational style after Carl Rogers, who introduced the use of open-ended questions to encourage patients to communicate more effectively with therapists. It is considered the forerunner of thinking machines. Weizenbaum was shocked that his program was taken seriously by many users, who would open their hearts to it. He started to think philosophically about the implications of artificial intelligence and later became one of its leading critics. source: wikipedia
u s e d
t h i n k
referred to a void or a gap. â¤ľ
empa r e a l i z i n g
nine years of my career, I apprenticed as a Computer Scientist and an Interaction Designer, under the teachings of both Human-Centered and InformationCentered1 design.
A house-style of design pioneered by Dr. Peter Lucas at m aya d e s i g n .
Introduction | space
During this time, I had a blast designing, programming, and interacting with
computers of various shapes and sizes, slowly making way for a new era where hundreds of different kinds of personal computers exist, each designed to serve a different context.
But for reasons I could not fully articulate, I wasnâ€™t satisfied with the product of either approach to design, and I was left with a feeling that s
In search of what
I sought an environment that would provide
a stark contrast to my own.
After a long period of search,
2008 I decided to spend the next three years immersed in an art school, engaged in
dialogue with physical materials.
Introduction | space
2009 a p e r
e t a l
f o l d i n g
p a p e r
n e w s p r i n t
c a s t i n g
e r i n g
p e n
t e e l
o o l s
p o p l a r
f i g u r e
d r a w i n g
t y p e s e t t i n g
b r o n z e
w a x
l e a d
c o m p r e s s e d
w o o d
w o o d w o r k i n g
g l a s s
d a n c i n g
w o r k i n g
h u m a n
g l a s s
b o d y
l e t
w i t h
h a n d t o
b o a r d
d r a f t
a c r y l i c
a c t i n
In school, I started out by working with paper, charcoal, clay, and plaster; then moved to type, metals, wood, m a r k e r s m o d e l m a k i n g gouache, glass, plastic, and light.c h i p
p l a s t i c
t y p e
c h a r c
p h o t o
d e v e l o p m e n t
c o a l
v i n e
t e r p r e s s
o o l s
i n g
2011 i n k
b a s s
l e a d
c h a r c o a l
w o o d
l i t h o g r a p h i c
s c u l p t i n g
p a i n t i n g
p l a s t i c i n e
g o u a c h e
c r a y o n
c l a y
s k e t c h i n g
p h o t o
i n d i a
p l a s t e r
r e n
m e t a l w o r k i n g i n • a l u m i n u m Along the way, I added acting and tdancing classes to my schedule as I came to realize the significance of the human body
v e l l u m
d i r e c t i n g
l i g h t
h u m a n
b o d y
c h e m i c a l s
as wa ophysical o d w o r material. k i n g w i
p h o t o
s e n s i t i v e
m a c h i n e
p a p e r
As I reflected upon these experiences through writing, I came to two realizations.
when we make things with physical materials, we learn how to adopt the perspective of the materials, to understand and respect their integrity, to make in collaboration with them, by engaging in an empathic conversation.
the spaces designed for working with these physical materials are designed to provide perspective, invite agency, raise self-awareness, and breed trust.
Introduction | space
Based on this insight, I came to the realization that the design of the modern personal computers lack
a deep sense of
empathy, and a firm grounding in ethics.
Empathy? With computers? I agree that the best developer is inevitably quite the computer
Hi. I’m An-Lon. I first met Slim back in 1998 attending the International School of Beijing. I think of him as Chan in the second person because that’s what he went by in high school, and djslim in the third person because I’ve only ever interacted with him online since. I guess everyone else calls him Slim. I haven’t quite wrapped my head around that yet. Over the years, I’ve spiralled from comparative literature to computer science to user interface design to computer graphics to character animation. While I’m a geek at heart, with my fair share of the stereotypical social inadequacies, pretty much every big break in my career has come from possessing some unusual degree of empathy, all the way from my amateur exercise in anthropology that drew me to geek culture, to my current foray into character animation, which is all about convincing audiences that dead pixels can walk, talk, laugh, and cry. The topic of computers and empathy is something I care deeply about.
whisperer, but I would have never actually thought of that rapport as empathy.
five necessary qualitiesâ€”
I then distilled and developed a list of
a set of shared metaphors, and a sense of trust, honesty, integrity, and dignityâ€Šâ€” that our interaction with the computer must afford before it can facilitate an empathic conversation between software computation and the human body.
Introduction | space
The outcome of this thesis exploration is a preliminary investigation into how theÂ language and the qualities of making things with physical materials can translate to the domain of computing.
In my work, I conduct this exploration using
a full-scale model,⤵ an animated short,⤵ an installation,⤵ a performance,⤵ video sketches,⤵ diagrams,⤵ and this book.
p. 266, 316
p. 92, 160, 206, 272, 324, 372, 410
( c ontinued from p. 59 ) Instead, the two synonyms that came to my mind were grokking and acculturation. Grok is a term
Conversation | space
Yes! I find the point you make about the relationship between empathy and the attainment of a sufficiently deep understanding to be critical. This was non-obvious to me in the past, but I am starting to realize that the ramifications of that statement could be as simple as supporting the fact that you cannot learn to ride the bike by merely reading about it or as complex as helping us understand why there is so much conflict in the world.
invented by science fiction author Robert Heinlein in st r a nge r i n a st r a nge l a n d,
and it’s taken on
a life of its own since. I’ve always taken it to mean a bone-level deep understanding of something. And that something was always a Computer Science concept, partly because the people who’d actually use the term were all geeks, and partly because so many concepts in Computer Science cry for a distinction between being conversant and being a guru. But I was surprised when I read the w i k i p e d i a entry on the term because it made me realize that such a deep understanding does indeed revolve around empathy, to the point where the o x f o r d dictiona ry
e ngl ish
actually defines to grok as
to understand intuitively or by empathy; to establish rapport with and to empathise or communicate sympathetically (with); also, to experience enjoyment. I had turned to w i k i p e d i a as a joke, but it did hit a nerve. The key takeaway, to me, is that it’s never a purely intellectual exercise to really truly understand something, be it an immediate piece of code or a deeper-underlying Computer Science concept.
Conversation | space
Yes! The example I often use to talk about myself empathizing with computation is the experience of debugging. When debugging, I am trying to figure out why the program is behaving the way it is, and my head gets filled up with nothing but my understanding of the state and configuration of the program. What Iâ€™m doing is essentially trying to think from the perspective of the program. And yes, I think programmers tend to have an enormous amount of patience and empathy for the computer, because they understand the principles by which they behave. When we lose patience is when we get the hour glass or the beach ball. Thatâ€™s
That reminds me of people telling me I have an extraordinary amount of patience in front of a computer. Do you get that? Is it common for geeks? I don’t know if it’s because I know very well what the computer is doing inside, I can be patient even if it’s slow or stalling. And also, whenever my dad encounters a computer problem, he always asks “How stupid is this computer! Why can’t it do this and that?” And I always feel like I’m defending the computer saying “It just can’t... this is what it can and cannot do. Don’t be too hard on it. Be patient... it’s still crunching numbers... There’s nothing you can do except rebooting the machine... And here’s a way to work around its
Hi. I’m Anson. I met Seung Chan in college. Although he was in Computer Science, and I was in Electrical and Computer Engineering, our dorms were close to each other, and we had a common interest in music and guitars. After college, I worked at b b n t e c h n o l o g i e s — n ow r ay t h e o n — as a Speech Software Specialist / Scientist. Much of my work involved language model training, pattern recognition, signal processing, and some h c i . Then about six years ago, I sensed a calling from God to become a pastor, and I quit my job to enroll myself into a Theological school. I have just finished my studies and now I’m a pastor at a church in Vancouver. So first, I am going to be up front about my faith and convictions. I hope to contribute my anthropological understanding of what it means to be human, for I believe Anthropology stems from Theology.
limitations . . . yada yada yada.” Isn’t that also related to empathy?
Conversation | space
actually more like a human being who is too pissed off to tell you what is actually going on.
Couldn’t agree more about the debugging. Perhaps we use our empathetic faculties for debugging because our brain cells weren’t equipped to access that deeper cloud of intuition any other way. Our brains have been wired for millennia to interact with fellow humans and, as far as I’m concerned, it’s an extremely useful act of hijacking to tap that empathic cloud in order to outsmart a machine. Just to be clear, I’m not talking about anthropomorphizing the machine, I’m talking about accessing our own preexisting, well-developed resource of empathic faculties to interact with it. Oddly, I anthropomorphize just about everything under the sun — teddy bears, disappearing keys, food that’s been in the fridge for too long. But I’ve pretty much never been tempted to anthropomorphize computers. Oh, and I also wanted to talk about the second word that came to mind in the context of computers and empathy: acculturation. Computers and computer languages are created by humans. The magic that bridges the abstraction of 0’s and 1’s with human neurons is language. Windows and mice are metaphors—picture a window in a house, picture a mouse running around on its four little paws, now marvel at the metaphor that got us where we are today! Zipping, unzipping, bugs are all metaphors. Computer concepts are all abstract until we give them names and map them to something we can understand. Even the act of stepping through code using a debugger is a concession to a human need for a linear storyline. Learning programming is similar to language acquisition, not because computer languages are
Conversation | space
Yeah, the role of the programming language designer has always fascinated me. Itâ€™s the authority figure who
anything like natural languages—university administrators who allow C++ to fulfill a language requirement like French or Spanish should be taken out and shot—but because learning how to write code is very much a process of acculturation. Just as it’s pretty much impossible—or at the very least, pointless—to learn a natural language without a cultural context, it’s impossible to write code well without absorbing its many sub-cultures. Best practices, conventions, idioms, and design patterns are all cultural constructs within a human community, not semantic ones within the machine. Put in that light, the idea of empathy with computers is staggeringly mundane — we’re talking about forming a rapport with the community of their very human creators, not a sentient and malevolent Hal. And yet, my rapport with, say, Linus Torvalds goes through multiple layers of translation, not the least of which is through the machine and back. And if I were to go out and write a Linux patch, I’d damn well better have empathy with the Linux Operating System so I can design something appropriate . . . and yet it doesn’t feel like real empathy. It’s not real the way a spoken word is real, a heartbeat is real, the touch of a hand is real. And yet, to anyone who’s ever gone into a programming trance and been absorbed to the point of forgetting to eat, sleep, or shower, it’s profoundly real, perhaps more real than reality. It’s an emotional state as much as a physical one. And here we cycle back to the initial conundrum: how to reconcile that austere landscape of programming abstractions with our emotional, embodied, messy selves — selves so much in need of human connection that we perhaps see everything through that lens.
Conversation | space
decides the syntax and grammar. It’s that person who secretly wishes to be God, but would probably never admit it. But, most of the ones I know are logicians who want things to be neatly categorized and exact. Why not a messy and evolving language? What is an evolving language? How does one design an evolving language? How do the principles of evolving systems translate to the design of programming languages? But more importantly, let’s talk briefly about metaphors here. As far as I can tell, the reason why metaphors work is because it triggers past experience, a physical response to a past experience, as opposed to simply a pointer to some abstract concept. In this sense, I find the current metaphor of the desktop or the window or any other similar shallow associations to be woefully inadequate. It promises such richness, yet falls so short in delivery. The only relationship between a real window and the idea used on the computer is that it puts a border around things. What happened to the feeling of space I feel when I look through a real window? It’s a bait and switch scam! People just don’t think of it that way.
I agree fully that the most salient and powerful computer metaphors come from those that most directly call to mind a tangible human experience. I’m really curious what your other friends with more of a formal background in some aspect of h c i think as well. But, just to clarify, I wasn’t commenting on the adequacy or inadequacy of windows/mice as a metaphor—it was more just a reminder that we really do fabricate computer concepts from human experience rather than out of whole cloth.
Conversation | space
Here’s a slightly loony bins example I have actually tried and failed many times to write down — it happened in the moment and I’ve never been able to replicate it to my satisfaction. Around the time I was learning Object-Oriented Programming — sometime in my early twenties — my cousin went through a love life crisis. The guy she was dating had a photo of an ex up on his refrigerator, but only her business card — no photo. They somehow got into a fight over this. She went home, and, partly out of pique but mostly to amuse herself, she got out a photo of every single one of her ex-boyfriends, put those photos on the fridge, and added the business card of the current guy. Then she forgot about it and went about her daily business. Of course, you can predict the rest of the story. The new guy somehow came over unexpectedly and saw the photos, they had another fight, and finally broke it off for real. The thing my cousin tried to explain to me later was that the problem wasn’t so much the photos and business cards and exes. It was that the guy just didn’t get that she does quirky things like that for her own amusement. It wasn’t intended as a message and wasn’t intended to be seen, it was just an expression of her own personal loopiness. The fact that he couldn’t relate to her silliness was as much the deal-breaker as the original photo of his ex. At the time, we were both pretty fresh out of college and lamenting the closeness of college friendships. The guy in question was older, maybe in his thirties — yeah, ancient, I know — and he really just didn’t seem to get it. And here is where I went into the spiel I have never been able to replicate since. The thought in my head — because I had just been reading about Object-Oriented Programming — was that in college,
Conversation | space
we gave out pointers left and right to each others’ internal data because we just didn’t know better. All the joy and sorrow and drama was there for any close friend to read — and write, and modify. As we got older, we learned that this is a rather dangerous way to live, and developed more sophisticated class interfaces — getters and setters for that internal data, if you will. The guy in my cousin’s story seemed to live by those getters and setters, and was appalled when my cousin inadvertently handed him a pointer. Here’s the part of the story I have never, ever been able to replicate: I told my cousin all that without mentioning Object-Oriented Programming once. I used a fair bit of Object-Oriented terminology, but only the words whose meanings were either immediately clear from the context or already in common usage — handle and interface, for example. She immediately understood what I was trying to say, and added that the word handle was a particularly poignant metaphor. When we’re young, we freely give loved ones a handle to our innerselves, but in adulthood, we set up barriers and only let people in at predetermined checkpoints according to predetermined conventions. As adults, we give out handles to only a very few, and those already in possession of a handle can always come back from a previous life to haunt us. We interact with the rest of humanity via an increasingly intricate set of interfaces. By now, I possess a much deeper and richer set of interfaces and protocols than I did in my early twenties, so I can share a great deal more of myself without fear of being scribbled on. But I still don’t hand out raw pointers very often — the vulnerability is too much for me, and the responsibility too much for the other person. Back to computers and h c i . I get scared sometimes by how often I use computer terminology in daily life among non-programmers and get away with it. You
Conversation | space
I’ve watched that Brené Brown talk numerous times. It’s a very good one. It is highly consistent with my experience making physical things. Humility and courage — what allows us to embrace vulnerability — are prerequisites to making something that has a soul. It is excruciatingly difficult to do, though. By the way, I’m really taken by your analogy of pointers and getters / setters in the context of
don’t have to be a programmer to understand me when I complain that an instruction manual is spaghetti, or that my memory of a particular song got scribbled on by someone else’s more recent cover of it. The reason these metaphors work, of course, is that spaghetti and scribble are essentially roundtripping as metaphors — f rom daily life to Computer Science and then back to daily life. First the English words were co-opted to convey a specific Computer Science concept — spaghetti code is code that is unreadable because it tangles in a million different directions, and to scribble on a memory location is to overwrite data you’re not supposed to overwrite — and then I re-co-opted them back into English — to express frustration at the unreadability of the instruction manual or lament that my memory of the original song has been tarnished. My point here is that Computer Science is rich in human meaning precisely because we choose human metaphors to express otherwise abstract concepts. My analogy between Object-Oriented Programming and human relations is surprisingly salient because Object-Oriented Programming, at some level, had to come from human experience first. What is architecture? It was the Sistine Chapel before it was the Darwin os. Have you seen the t e d talk by Brené Brown on the power of vulnerability? It’s what got me thinking about our longing for human connection.
Conversation | space
a relationship. I’ve never thought of it that way, and it’s a rather interesting way of thinking about it. I’d like to talk more about this later in relation to learning how to act, but I think it’s worth talking more about the idea of humanness right now. It seems that back in the 1950s, a group of Computer Scientists wondered if computers could ever be like human beings. This question lead them to start asking an endless stream of questions trying to understand the human mind like what makes us human? How does thinking work? What is consciousness? How do we learn? How do we understand something? Where does intelligence come from? These were absolutely fascinating questions, and we still have not found any definitive answers to any of them. Where things get less interesting, for me at least, is when they started asking how we can abstract the humanness from ourselves, to disembody it, so as to put it in some other body, and debate whether that other being is also human. Honestly, I cannot see why this line of questioning is valuable. What this kind of disembodied attempt at manifesting humanness can do, at best, is superficially mimic or simulate what one may mistakenly believe a human being to be, without any real understanding of what it actually means to be human. But, I don’t think this is to say that one cannot feel for the computer. In fact, having spent a significant portion of my career programming, I have often found myself totally immersed in thinking from its perspective and not mine. And qualitatively speaking, I think this is analogous to the feeling of empathizing with another human being. Most think we only empathize with “living things” that “feel”, but I know people who tell me they empathize with plants. Some with trees or even dishes. It seems to
Conversation | space
me that people have varying capacities for empathy. To claim that empathy only pertains to living things that feel does nothing but limit the potential to fully understand the human capacity for empathy, and the extent to which it is utilized in a variety of contexts.
Hmmm... I never thought of this as non-secular. Metaphysical, yes. Are you equating the two?
Slim, how are you going to treat this subject in a secular fashion? It’s going to be very difficult, because so much of it leans toward feeling and emotion as opposed to logic, science, neurons, etc. It barks up
Hi, I’m David. I know Slim from working with him for a couple years at m aya d e s i g n in Pittsburgh. I work in software, and I’m also a musician, a photographer, a skier, a cyclist, a runner, a thinker, a reader, and a writer. I have a deep need to understand reality in its purest form, to seek the highest levels of production quality even when they don’t matter to anyone but me, and to achieve symmetry in literally everything. They say that at the root of engineering is this truth seeking and you’ll see elements of that here from me.
the quality / q uantity tree that is split down the center and very divisive.
Well, just as the z e n
a nd the a rt
of motorcycle m a i nte na nce
not about Zen, I don’t think what you’re after is about empathy. It’s deeper than that. To empathize with the computer is to anthropomorphize. To anthropomorphize is to visit our expectations on reality. Computers aren’t humans and they never will be. Can man make a better human? Probably. Will that human have better distinguishing human characteristics? No. In the very same sense that James Howard Kunstler argued that architecture was moving toward a loss of a sense of place — which I agree with — robotics goes down the same
dav i d wat s on
Conversation | space
What do you mean?
boring path, most likely because it has no other choice. If it didn’t, we’d be defining an engine of individuation, and I’m pretty sure nobody is doing that. And that’s the miracle of humanness.
dav i d
Speaking of humanness, historically speaking, Artificial Intelligence referred to abilities that we thought computers were incapable of. However, as solutions to problems began to appear, these abilities — like chess playing and language recognition — were no longer considered intelligence. How computer programs tackle intelligent tasks is always different from how humans actually do them;
Hi I’m Jeff. I know Slim from when he was a workingman on the South Side of Pittsburgh at m aya d e s i g n and I was a Ph.D. student in h c i at Carnegie Mellon. I will be bringing my background in Computer Science and Cognitive Science along with perspectives from theoretical Computer Science, conceptual history of a i , Psychology, and Cognitive Science. I am also familiar with Psychiatry, Phenomenology, and tidbits of religion and spirituality along with a dabbling in Philosophy.
sometimes better or more thorough, but at other times, seemingly stupid.
Why does a i trip up on “special cases”? Because the way we program intelligence is by making problems formal — i.e. accessible to the computer. When problems are formalized, they can be solved by rules. Where rules don’t quite work, we have rules for selecting rules, or rules for creating the rules to select rules with — i.e. machine learning. I think we approach problems this way because our way of accessing how we think, and communicating that to other people is in the framework of rationality.
Conversation | space
I think rationality is primarily a structure for thinking about thinking. How we think isn’t quite rational. It’s more like rational++. When people appear rational, we can empathize with them. Irrational people, too, but not as easily. Part of anger is not knowing why. If you know how the machine works, you can be angry at the situation, but it’s not the machine’s fault, it’s whatever is broken or not working inside. Does anger require a thing to be angry at? If you’re angry at the thing and you don’t know how it works, it might as well have a mind of its own. It makes no difference to you. Consider the University of Texas clock tower shooter. You can imagine being angry at him for shooting someone you know, but then you find out he had a tumor and he requested an autopsy in a note he left at his house with his dead family. Somehow that kind of situation is a bit less angry because you know why. Understanding the mechanism changes how we think about the thing. I remember being excited about taking an a i class and learning the magic. But it turned out be a whole bunch of hacks — or so it appeared. It’s no longer magic when you know how it works. Now, I don’t quite understand what you mean by empathizing with computers. What you’re doing when programming is simulating your program on your model of the programming language runtime. Yes, it’s sort of like empathy, but I thought empathy was I can see how you feel that way. We empathize with real people based on our concepts and experiences of other people. This is 1
the Theory of Mind , which autistic people lack. For that, they are alone in the universe because other people simply don’t exist. So you need models of other people to empathize with them.
Theory of mind is the ability to attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have beliefs, desires and intentions that are different from one’s own. Theory of mind appears to be an innate potential ability in humans, but one requiring social and other experience over many years to bring to fruition. Different people may develop more, or less, effective theories of mind. Empathy is a related concept, meaning experientially recognizing and understanding the states of mind, including beliefs, desires and particularly emotions of others, often characterized as the ability to “put oneself into another’s shoes.” source: wikipedia
Conversation | space
Yes, I will. But, allow me to first write about my experience empathizing with physical materials. Because thatâ€™s where this train of thought started from. I will also talk about my experience with a friend who had bipolar disorder. It is the qualitative comparison between my interaction with physical materials and my experience with my friend that solidified my theory. God! I canâ€™t tell you all how much I love this tightlyknit discussion environment!!!! :)
Still, empathy is a feeling about feeling. I still don’t get empathizing with computers. My current idea of what you might be thinking seems wrong.
I second Jeff on this point. I don’t quite understand what it means to empathize with computers, either. I don’t necessarily think that you need to have answers to all these questions now. Some of them are certainly empirical questions, and worth investigating more. But, I would still like to have a better grasp of your idea of empathizing with computers, and
Hi, I’m Joonkoo, and I know Seung Chan hyung from Beijing ( hyung means older brother in Korean ) where I attended the International School of Beijing with him. I am now in the final stages of my Ph.D. studying Cognitive Neuroscience of high-level vision. Most of my work is centered around neural organization and mechanisms of object recognition, such as faces, letters, and numbers. However, I’m getting more interested in numerical cognition, as I plan to study the neural basis of number sense during my post-doctorate studies. While I am trained as an experimentalist and my interest is pretty focused, any questions related to how the mind works triggers my interests. I wish to bring in some nonscientific and psychological ideas into the discussion.
I still can’t quite get it. Perhaps it will get explained in your future writings?
joon koo pa r k
Glossary | space
(continued from p. 43) But now I have now come to think of space, instead, as a means to occupy. To occupy implies taking a position, a stance, a perspective. Occupying a space allows us to take on an occupation, as a woodworker working in a wood shop, or a blacksmith in a foundry. In computation, we occupy a disembodied space. And as a result, the space of computation is reflective, much like a mirror. Through it, we can see our disembodied selves reflect back at us, giving us a perspective on ourselves that we may not have had otherwise. I have also come to think of space as a medium for acting. Without space, we wonâ€™t be able to act. Without space we wonâ€™t have agency. In a computational space, our cursors move as a proxy of our minds. And in doing so, the space of computation works once again like a mirror. Through it, we can see our desires for action reflect back at us, and manifest at a distance.
Glossary | space
Finally, I have come to think of space as that which completes the self. Space allows us to distinguish between the self and the other, between the self in the present and the self in the past. In the space of computation, we become aware of where our body ends, and the mind continues. And we can lose sight of where one mind ends, and another begins. But whatâ€™s important
computation is space, then the computer is the instrument that bridges the space of human body and the space of computation. I know many of us think of computation as merely a means of simulation, calculation, or distribution. So, for the remainder of this book, I would like you to seriously consider what it would mean to think of computation as space. A space with the potential to give us perspective, invite agency, and raise self-awareness.
minds extend into. And if
our desires act in, and our
is that computation is space. A space our identities occupy,
C OM P UTER
ph ysic s a s fr eedom (20 09) is an animated feature exploring our relationship to both computational materials and physical materials. The film points to the concept of freedom put forth by the early pioneers of personal computing such as Richard Stallman, and reframes it through our freedom to enact an everyday verb such as “to cut” on a piece of paper. By illuminating our lack of freedom to enact such a seemingly simple gesture on a piece of digital document, the film challenges you to critically rethink the nature of your relationship to personal computing.
seung chan lim
vimeo.com/8184534 mot ion pic ture
2 min 25 sec
Much of what we do everyday involves re-enacting verbs on things. For example, trimming a piece of paper may be described as reenacting the verb cut on the piece of paper.
Just as we reenact verbs on things, we often times reenact verbs on files.
But for the average person, reenacting verbs on files tends to involve many more steps, and takes a much longer time to complete.
But the real difference is that in the physical world, if we donâ€™t have a knife to cut with, we can choose to use our own bare hands. If our hands are tied up, we can even use our teeth. This is freedom.
The computer is a “psychological machine.” On the border between mind and not mind, it invites its anthropomorphization, its psychologization. It does this almost universally, for children and grown-ups, men and women, novices and experts. This does not mean that people see it as “alive,” but rather, there is a pull to psychologize the machine, to give it an intellectual and aesthetic personality. The computer facilitates a relational encounter with a formal system. Sherry Turkle / s o c i o l o g i s t ,
As an Abby Rockefeller Mauze Professor of the Social Studies of Science and Technology at the Massachusetts Institute of Technology and a sociologist, Sherry Turkle has focused her research on psychoanalysis and culture and on the psychology of people’s relationship with technology, especially computer technology and computer addiction. Turkle also explores the psychological and societal impact of such “relational artifacts” as sociable robots, and how these and other technologies are changing attitudes about human life and concepts about what it means for something to be alive. One result may be a devaluation of authentic experience in a relationship. Turkle was formerly married to Seymour Papert, and together they wrote the influential paper “Epistemological Pluralism and the Revaluation of the Concrete.” source: wikipedia
u s e d
/risk/ risk t h i n k
lead to permanent and fatal damage. â¤ľ
â¤ľ p. 157
e m pat h y
a n d
Could you please introduce yourself,and talk briefly about your area of expertise?
I understand that your research deals with the process of learning and risk taking. Could you start by
taking Yeah, I am Lewis P. Lipsitt, Professor Emeritus of Psychology, Medical Science, and Human Development at b r o w n u n i v e r s i t y.
I have been at
brow n uni v e rsit y
working in the field of Child
An American Professor of Psychology who focuses his major research interests on infant behavior and development, the effects of early experience on later development and behavior, perinatal risk factors and their role in developmental destinies, crib death, and adolescent risk-taking behavior. The threads running through all of these pursuits relate to learning processes and the role of pleasure and annoyance (reinforcement features) as determinants of learning. s o u r c e : l i f e s pa n . o r g
Psychology Development, particularly Infant Behavior and Development.
Pages 112 â€“ 407 are not included in this limited preview. For a full copy of the book, please visit: http://realizingempathy.com/#writings
Bloom, Harold. s h a k e s p e a r e : t h e i n v e n t i o n New York: Riverhead Books, 1999.
McCloud, Scott. u n d e r s ta n d i n g c o m i c s : New York: HarperPerennial, 1994.
t h e i n v i s i b l e a r t.
Bohm, David, and Lee Nichol. On Dialogue. London: Routledge, 1996.
Noë , Alva. o u t
of our heads
why you are not
your br ain, and other lessons from the biology
Capra, Fritjof. u n c o m m o n with remarkable people.
: c o n v e r s at i o n s Toronto New York: Bantam
New York: Hill & Wang, 2010.
Ortony, Andrew. m e ta p h o r Cambridge UP, 1993.
Connor, Richard. u n d o i n g
New York, NY: Little, Brown and Co, 2010. Crawford, Matthew. s h o p
: New York: Penguin
class as soulcraft
a n i n q u i r y i n t o t h e va l u e o f w o r k .
Books, 2010. : Urbana: University
a r t c a n n o t b e ta u g h t
a handbook for art students.
of Illinois Press, 2001. Feynman, Richard. t h e
: thoughts New York: Basic Books, 1998.
meaning of it all
o f a c i t i z e n s c i e n t i s t.
Frost, Robert. “Education by Poetry.” Amherst Graduate Quarterly 20.2 (1931): 75-85. Gantt, W. Horsley. “On Humility in Science.” Integrative Physiological and Behavioral Science 26.4 (1991): 335-38.
Harvey, David. s p a c e s
Pangaro, Paul, Michael Geoghegan, Hugh Dubberly, Peter Esmonde, n o t e s o n t h e r o l e o f l e a d e r s h i p a n d l a n g u a g e i n r e g e n e r at i n g o r g a n i z at i o n s . Sun Microsystems, 20 02 Pelletier, Louise. a r c h i t e c t u r e , e t h i c s , a n d t e c h n o l o g y . M ont r é al M ont r e al Buffalo: Ins t i t ut de r echerche e n h i s t o i r e d e l ’ a r c h i t e c t u r e , Canadian Centre for Architecture, McGill University McGill-Queen’s University Press, 1994. Penny, Simon. “ c o n s u m e r
: t o wa r d s New York: Routledge, 2001.
c u lt u r e a n d t h e
t e c h n o l o g i c a l i m p e r at i v e : t h e a r t i s t i n d ata s p a c e .”
1993. Web. 14 February 2010. <http://ace.uci.edu/ penny/texts/consumerculture.html/>. Proust, Marcel, Andreas Mayor, Terence Kilmartin, D. J. Enright, Joanna Kilmartin, and Terence Kilmartin. t i m e r e g a i n e d . New York: Modern Library, 2003. Rifkin, Jeremy. t h e
Hamming, Richard W. “You and Your Research.” Bell Communications Research Colloquium Seminar. Morris Research and Engineering Center, Morristown. 7 Mar. 1986.
a c r i t i c a l g e o g r a p h y.
a n d t h o u g h t.
w h at t h e r a p y
d o e s n ’ t t e a c h y o u a n d m e d i c at i o n c a n ’ t g i v e y o u .
Elkins, James. w h y
e m p at h i c c i v i l i z at i o n
the r ace to
global consciousness in a world in cr isis.
J.P. Tarcher/Penguin, 2009. Sen, Amartya. d e v e l o p m e n t Anchor Books, 2000.
o f c a p i ta l
Harvey, David. a b r i e f h i s t o r y o f n e o l i b e r a l i s m . Oxford New York: Oxford University Press, 2007. Hagen, Uta. a c h a l l e n g e f o r t h e a c t o r . New York Toronto New York: Scribner’s Maxwell Macmillan Canada Maxwell Macmillan International, 1991. Kramarae, Cheris. Technology and Women’s Voices: Keeping in Touch. New York: Routledge & Kegan Paul, 1988.
Simon, Herbert. t h e s c i e n c e s o f t h e Cambridge, Mass: m i t Press, 1996.
art if icial.
Stapleton, Laura M. Toward Present Listening: Practices and Verbal Response Patterns in Small Groups of Teacher Candidates and University Supervisors. Thesis. Stanford University, 2008. Tarkovsky, Andrey. s c u l p t i n g i n t i m e : r e f l e c t i o n s o n t h e c i n e m a . Austin, TX: University of Texas Press, 1987.
Laing, R. k n o t s . New York: Vintage Books, 1972.
Weizenbaum, Joseph. Computer Power and Human Reason: from Judgment to Calculation. Harmondsworth, Middlesex, England: Penguin, 1984.
Shannon, Claude. t h e m at h e m at i c a l t h e o r y o f c o m m u n i Urbana: University of Illinois Press, 1998.
b r a i n , l a n g u a g e , a n d h u m a n c u lt u r e .
c at i o n .
Wilson, Frank. t h e
how its use shapes the
Vintage Books, 1999. Kearney, Richard. d i a l o g u e s w i t h c o n t e m p o r a r y : the phenomenological h e r i ta g e : p a u l r i c o e u r , e m m a n u e l l e v i n a s , h e r b e r t m a r c u s e , s ta n i s l a s b r e t o n , j a c q u e s d e r r i d a . Manchester, uk Dover, N.H., USA: Manchester University Press, 1984.
c o n t i n e n ta l t h i n k e r s
Vesely, Dalibor. a r c h i t e c t u r e i n t h e a g e o f d i v i d e d : t h e q u e s t i o n o f c r e at i v i t y i n t h e s h a d o w o f p r o d u c t i o n . Cambridge, Mass: m i t Press, 2004. r e p r e s e n tat i o n
Yam, Yaneer. m a k i n g Markoff, John. w h at t h e d o r m o u s e s a i d : h o w t h e s i x t i e s c o u n t e r - c u lt u r e s h a p e d t h e p e r s o n a l c o m p u t e r i n d u s t r y . New York: Penguin Books, 2006.
: s o lv i n g c o m p l e x Cambridge, MA: NECSI,
problems in a complex world.
Knowledge Press, 2004.
model of empathic con v er sation
( 2011 )
courage N PRI
D U N
B O E M
seung chan lim
TA N D I N G
O W L E D G E
humility L E D G E
I N G
PR NCI I
RS U N D E
trust honesty integrity dignity
I E O D E M B
S T O RY O F WH O W E ARE
on the web: