Forgetting To Remember: Reconnecting Our Artifacts And Histories by Ben Creek

BENJAMIN CREEK s3083541

[bits] Interactive Ecologies RMIT Industrial Design . Pre-Major Project 7 2011

FORGETTING TO REMEMBER RECONNECTING OUR ARTIFACTS AND HISTORIES

CONTENTS INTRODUCTION

PROJECT STATEMENT

ABSTRACT

RESEARCH & INFLUENCES FIELDS OF STUDY CULTURAL THEORY

9 10

TECHNOLOGY

PRECEDENTS

PAST INVESTIGATION

IDEATION

EXPERIMENTS

GRAPHICAL REPRESENTATION

IR TOUCH SCREENS

PROJECTION PROJECTION MAPPING INTERACTIVE ZONES

PROPOSED OUTCOME

OVERVIEW

CONCEPT SCENARIO

DEVELOPMENT

TIMELINE

74 79

INTRODUCTION

PROJECT STATEMENT

In a world where we are increasingly alienated from our material artifacts and histories, how can we reconnect to these lost cultural signifiers in our personal lives?

ABSTRACT This project looks at the effect of impermanent artefacts on our memories and what we can do to prompt us to recall these connections in the domestic environment. We live in a digital world filled with multifunctional devices and transitory objects. Obsessed with visual culture, we are heading towards a post-object existence. In such a scopophilic society vision is privileged and we become adept at looking, but this has lead to a loss of connection to our histories and artifacts on several levels. Most obviously, we have lost many of these visual signifiers from our daily lives as we minimize the range of objects that we possess. Multi-functional and multi-purpose devices have reduced the number of personal and domestic tools we use and the consumption trends of late capitalism see us transitioning through the objects we do possess at an alarming rate, preventing us from attaching memories, meaning and significance to them. Paradoxically, as we reduce this ongoing connection to these things, we concurrently become inundated with images and representations in this postobject society and often struggle to cope with the sheer volume of digital information that we possess.

This project attempts to harness our short attention span for visual culture, our tendency to glance, in a system that reintroduces these connections to histories and artifacts. By providing a better link to histories and artefacts, we may be able to lead a more enriching life. based on the initial research performed, the most successful outcome for this project will involve the creation of an interactive system that utilises visual projection to embed a virtual layer onto physical space. This system will focus on sight and touch as the core feedback modalities.

RESEARCH & INFLUENCES

FIELDS OF STUDY

PSYCHOGEOGRAPHY & THE FLANEUR Psychogeography explores how elements of the built environment can affect a personâ&#x20AC;&#x2122;s behavior and their perception of that space. I was drawn to these areas with an interest in exploring whether the ways in which we navigate physical space might also apply to the ways we navigate and visualize our digital environments. Since these digital environments have affected how we store, view and share much of our memory stimuli (such as digital photographs and videos), we may be able to apply similar theories to the digital environment that the Situationists applied to physical space. The FlĂ˘neur - that particular kind of drifter described by Charles Baudelaire - is simply an individual who strolls around the city in order to experience it. This intuitive process could be used to find more playful and expressive ways of connecting to our histories in a system which explores the signifiers that spark our memories.

CABINET OF CURIOSITIES

ROMAN ROOM MEMORY TECHNIQUE

A cabinet of curiosities was an encyclopedic collection of objects whose categorical boundaries were yet to be defined that existed from the Renaissance through to the Victoria Era. They were also known by various other names such as Cabinets of Wonder and Kunstkammer or Wunderkammer in German (art-room or wonderroom). These spaces were in many ways the precursors to museums and galleries, presenting the viewer with a loosely arranged collection of artifacts meant to spark imagination, memory and mystery.

The Roman Room is a memory technique that works by coding information through attaching it to memories of known objects in a room or several rooms. The technique is an ancient and effective way of remembering unstructured information where the relationship of items of information to other items of information is not important. It functions by imagining a room, such as your bedroom, and associating an image with each of the objects that you know to be in that room. In order to recall information, one simply takes a mental tour around the room, visualizing the known objects and their associated images. The technique can be expanded by going into more detail, and keying images to smaller objects. Alternatively you can open doors from the room you are using into other rooms and use their objects to expand the volume of information stored. Different rooms can be used to store other categories of information. The technique can be applied to other spaces as well, with some users having envisioned a view or a town they know well and populated it with memory images.

BLOGS: THE NEW CABINET OF CURIOSITIES Several internet bloggers describe their sites as Wunderkammer, either because they are primarily made up of links to random things that are interesting or because they inspire wonder in a similar manner to the original Wunderkammer. Interestingly, blogs function

as digital Wunderkammer and are therefore removed from the physical signifiers that they represent. Robert Gehl describes internet video sites like YouTube as modern-day Wunderkammers and discusses how they too might be refined into capitalist institutions, â&#x20AC;&#x153;just as professionalized curators refined Wunderkammers into the modern museum in the 18th century.â&#x20AC;?1

This overlaying of information onto mental maps of geographic space overlaps with many ideas explored in the area of augmented reality (which is explored in more depth below) and also illustrates the brains ability to store spatial information and trigger memory data through geographic space information.

1 http://en.wikipedia.org/wiki/Cabinet_of_curiosities

SOUVENIRS - OBJECTS & PHOTOGRAPHS

DESIRE LINES The term “desire lines” was first developed by French Philosopher and poet Gaston Bachelard in his book The Poetics of Space. The term refers to a path developed by erosion caused by animals or humans walking along an unmarked path. They are a common sight with a loosely defined line offering an alternative path to the predetermined path the council had commissioned. The paths take on an organic appearance by being unbiased toward existing constructed routes. These are almost always the most direct and the shortest routes between two points, though not always the routes anticipated by urban planners. The phenomenon has in fact been used by urban planners in places such as Finland where planners are known to visit their parks immediately after the first snowfall when the existing paths are not visible. Navigating space contrary to the presanctioned manner is nothing new, but I find that with much of my digital environments the paths I have available to navigate and explore the information can be rigidly defined and less than intuitive Personal digital archives, such as photo albums and home videos collections, which many people use to connect with their histories, are a logical playground for these ideas to be implemented.

Deriving from the French word for ‘to remember’, a souvenir is a memento, keepsake or token of remembrance acquired for the memories the owner associates with it. The term souvenir brings to mind the mass-produced kitsch that is the main commodity of souvenir and gift shops in many tourist traps around the world, but a souvenir can be any object that can be collected or purchased and transported home by the traveller. A souvenir as an object in itself has no real significance other than the psychological connection the possessor has with the object as a symbol of past experience. Without the owner’s input, the object’s meaning is invisible and cannot be articulated. With many souvenirs mass-produced in places that have no relation to the tourist site beyond the image represented in the object, the signifier role of the object is reinforced as its worth is predominately determined by its relationship to its owner. The photograph would have to be the most common form of souvenir, serving as a medium to document significant events. The photograph itself carries much theoretical baggage with it when considered as an object of discursive analysis, particularly in relationship to ‘seeing’ and the role of the viewer. The question of the photographs status as an ‘original’ or ‘copy’ is also at play as with the photos ability to substitute a tangible object within the role of stimulating memory. As with our increasing consumption of physical objects, our dramatic increase in consumption of digital objects has been aided by developing technologies. Our ability to capture magnitudes of photographs with the advent of the digital camera and the ability to store them permanently with cheap data storage has revolutionised the photography industry. Without film restricting our capacity to take photographs or the limitations of physical storage space, the photographer can store an almost infinite number of images. With so many images in our possession, it becomes difficult to develop meaningful relationships to these objects individually, and we become attached to the collection - and collecting more broadly - rather than having intimate relationships to a special few.

PLANNED OBSOLESCENCE AND MASS CONSUMPTION Planned obsolescence is a policy or process employed by industry where companies deliberately plan or design a product with a limited useful life, so it will become obsolete or nonfunctional after a certain period. This puts the consumer under pressure to purchase the product again, either from the same manufacturer or from a competitor, creating artificial demand in that industry. Planned obsolescence was first developed in the 1920s and 1930s when mass production became the industry norm. Since this period and particularly since the post-war boom - mass consumption has also become normalised. Henry Ford, the father of mass production, and his peers understood that mass production would necessarily lead to mass consumption, and a range of tools have been used to ensure this continued relationship, including advertising, planned obsolescence and a culture of competitiveness.

SCOPOPHILIA AND GLANCE CULTURE

Scopophilia is literally “a love of looking”. Many theorists, including Roland Barthes, have argued that we live in a visual culture where vision is privileged above all other senses. However, in this image saturated world, while we have become extremely visually literate, we are also inundated with so many images that many of them don’t register on more than a superficial level. This has led to a glance culture where we no longer pay strict attention to the images before us, often at our own expense. For example, it has been said that the average time spent before artworks in a gallery or museum is less than 8 seconds.

ALIENATION Alienation is a process identified by Karl Marx whereby people become foreign to the world they are living in. Individuals often seek to mitigate their alienation through the aquisition of goods in the hope of buying their way out of their alienated state. It has been argued that by late capitalism the whole sphere of personal consumption had been reorganised according to commercial principles where cultural products also gain “a life of their own” completely independently of the producers.

PROJECTION MAPPING 3D projection is a method of mapping threedimensional points to a two-dimensional plane. An extention of this principle is projection mapping, a technique of beaming video (with a standard video projector) onto three dimensional objects and adjusting and masking the image so that it follows the contours of the shape of the target object. The result can be surprisingly effective and eye catching as the video is no longer a flat square on the wall but becomes an object in space - in effect, an animated sculpture. Projection mapping isnâ&#x20AC;&#x2122;t a new concept, but with the increase in cheaper technology capable of creating this effect, the technique is becoming more commonplace. It has come to be used more and more in commercial contexts, but the modding and artist communities still often use the technique in unconventional and interesting ways. FOLLOWING FORM Artist Barbra Kruger is famous for her use of type-immersed spaces which literally skin gallery spaces with text, making the surfaces of the space into the works themselves. Designer Ian de Grucci worked with Kruger on her show at ACCA, helping her transpose her text works into mapped projection.

AUTO CALIBRATION - PROJECTOR-BASED LOCATION DISCOVERY AND TRACKING

MULTI -POINT INTERACTIVE WHITEBOARDS USING THE WII REMOTE

Johnny Chung Leeâ&#x20AC;&#x2122;s PhD study at Carnegie Mellon University in the Human-Computer Interaction Institute employed the basic idea of using projected light to discover the locations of optical sensors. This location data can be fed back to the computer for use in a projected application. This results in a significant simplification and enables interactive projection and augmented reality applications. It eliminates the needs for calibration and eliminates the needs for an external tracking technology.

Since the Wiimote can track sources of infrared (IR) light, you can track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can create very low-cost interactive whiteboards or tablet displays. Since the Wiimote can track up to 4 points, up to 4 pens can be used. It also works well with rear-projected displays.

FOUR POINT CALIBRATION SURFACE MAPPING Using the Wiimote to track sources of infrared (IR) light, it is possible to track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can run software to calibrate the area by mapping it to a grid using the four point calibration method formalised by Johnny Lee.

WATCHOUT - MULTI-DISPLAY PRODUCTION AND PRESENTATION SYSTEM Dataton WatchOut is the leading multi-display production and playback system. The software based system can be used to orchestrate stills, animations, graphics, video, sound and live feeds in a single impressive show across multiple display areas, soft-edge or scattered. RMIT has a 3 projector screen Watchout setup installed in building 8 which i hope to use for simulating and testing my project. 25

AUGMENTED REALITY

MULTIPLE INFORMATION SPACES

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual computer-generated imagery.

It is possible to create multiple information spaces whereby some digital information sits outside of the throw of a projector. Rather than changing the canvas to fit the viewing space, it is possible to change the position of a portable or micro projector to reveal information displayed outside the original position. This creates an illusion of exploring large information spaces embedded in the physical environment as if using a flashlight.

AR systems typically generate a composite view for the user that is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. The virtual scene generated by the computer is designed to enhance the user’s sensory perception of the world they are seeing or with which they are interacting.

Using this technique, the display and interaction space can be expanded to cover almost an entire physical environment and support interaction concepts that are not possible on traditional desktop or handheld devices. Building on these unique affordances of handheld projectors, Xiang Cao and Ravin Balakrishnan have explored techniques for dynamically defining and interacting with multiple information spaces embedded in the physical environment using projection. Enriching the interaction possibilities and leveraging human ability to perform bimanual tasks they used a passive pen to support annotations and local interactions.v

VIDEO PAINTING SWEATSHOPPE Multimedia performers Sweatshoppe have recently been pasting buildings with moving images all over New York. Mapping video projections to LED-lit paint rollers, Sweatshoppe lay their projections on a surface, paint-stroke by paint stroke. They call this new digital performance style “Video Painting”. How it works: The software controlling the video was written in Max. The paint roller does not use any sort of paint, it simply contains green LEDs. The software tracks the color green and outputs the ‘x’ and ‘y’ positions which are sent to drawing commands and the strokes are textured with video. Sweatshoppe is video artists Bruno Levy and Blake Shaw. They plan on eventually releasing the software, but only after it is much more refined, buffed up with features and is userfriendly.

PERPETUAL STORYTELLING APPARATUS JULIUS VON BISMARCK & BENJAMIN MAUS

WANDERING IN KNOWLEDGE UNIVERSITY OF THE ARTS BREMEN

The “Perpetual Storytelling Apparatus” is a drawing machine that illustrates a never-ending story by translating words of text into patent drawings.

In the centre of the 15 metre high stairway at the University of the Arts library in Bremen, a sculpture of folded paper demonstrates the connection between the traditional storage medium and the digital world.

Seven million patents — linked by over 22 million references — form the vocabulary. By using references to earlier patents, it is possible to find paths between arbitrary patents. They form a kind of subtext. The machine attempts to show how new visual connections and narrative layers emerge through the interweaving of the story with the depiction of technical developments. Basic procedure:

This system projects automated graphics that reflect search results that have been entered into the library catalogue at that point in time. The media sculpture highlights the number of mental processes which take place simultaneously in the library. “Altogether the media installation poses questions about the function of the information in the age of the increasing communicational isolation. In regard to the title, the visitor literally passes through the world of knowledge.”1

1. The program downloads and parses a part of the text of a recent best-selling book. 2. The algorithm eliminates all insignificant words like “I”, “and”, “to”, “for”, “the”, etc. The remaining words and their combinations are the keywords for the patent drawings. 3. Using the keywords in chronological order, it searches for the key-patents. 4. The program now searches for a path connecting the found key patents. This is possible because every patent contains several references to older patents – the so-called “prior art”.1 5. All key-patents and the patents connecting them semantically are arranged and printed. 6. Repeat step 1.

1 http://www.creativeapplications.net/objects/perpetualstorytelling-apparatus-objects/

1 http://vimeo.com/13544405 http://wandernimwissen.wordpress.com/

VIRTUAL GRAVITY SILKE HILSING

APARTMENT MARTIN WATTENBERG & MAREK WALCZAK

“Representing the importance and popularity of search terms in the form of physical weight.”1

In Wattenberg and Marek’s Apartment, viewers are confronted with a blinking cursor. As they type, rooms begin to take shape in the form of a two-dimensional plan, similar to a blueprint. The architecture is based on a semantic analysis of the viewer’s words, reorganizing them to reflect the underlying themes they express. The apartments are then clustered into buildings and cities according to their linguistic relationships.

Virtual gravity is an interface between the digital and analogue world. Built using Processing, Arduino and reacTIVision, the installation allows you to measure weight (ie. popularity) between terms via “loading” the information into pads which are then positioned on mechanical scales. Terms that “weigh” more send messages to Arduino controlled cylinders which lower and rise depending on the term’s popularity in Google Search Insight. You can search desired terms via keyboard input operated via the same pads. In this way impalpable, digital data gets an actual physical existence and becomes a sensually tangible experience.

Each apartment is translated into a navigable three-dimensional dwelling, so contrasting between abstract plans/texts and experiential images/sounds. Apartment is inspired by the idea of the memory palace. In a mnemonic technique from a prePost-It era, Cicero imagined inscribing the themes of a speech on a suite of rooms in a villa, and then reciting that speech by mentally walking from space to space. Establishing an equivalence between language and space, Apartment connects the written word with different forms of spatial configurations, similar to teh Roman Room technique.

1 http://www.creativeapplications.net/processing/virtualgravity-processing/ http://www.silkehilsing.de/

BERT BONGERS INTERACTIVE READING TABLE Reading Table uses physical objects that people interact with to generate particular new media content such as videos, sounds and web sites. It combines the tangibility and clarity of printed media such as books, papers and articles with the malleability and flexibility of new media. The aim is to create an integrated experience for the users, bringing together traditional media (such as books) with new media (such as video). The table uses RFID technology to link the physical objects to the media content.

INTERACTING WITH DYNAMICALLY DEFINED INFORMATION SPACES USING A HANDHELD PROJECTOR AND A PEN XIANG CAO, RAVIN BALAKRISHNAN The recent trend towards miniaturization of projection technology indicates that handheld devices will soon have the ability to project information onto any surface, thus enabling interfaces that are not possible with current handhelds. Cao and Balakrishnanâ&#x20AC;&#x2122;s project interacts with multiple virtual information spaces embedded in a physical environment using a handheld projector and a passive pen tracked in 3D. They developed techniques for defining and interacting with these spaces, and exploring usage scenarios.1

1 http://www.cs.toronto.edu/~caox/uist2006_handheldprojector.pdf

PAST INVESTIGATION

INTERACTIVE WALLPAPER In 2010 as part of the Touch Points upperpool studio I started my inquiry into how projection touch technology could be ultilised in a domestic environment. The project aimed to examine notions of the home and how an alternative interface could be utilised to enrich the lives of a house’s inhabitants. The concept developed using projected imagery and touch surface technology to create a system which allows users to explore media in a more abstract and, in many ways, more intuitive method than those accommodated by current household computers. It was hoped that through this alternative interface users would find their memories stimulated in a unique manner and thus expand their sense of self, improve optimism and aid their social interaction. This project focused on the question: “how can we make a house a home?”. Alain De Botton, in his popular philosophy series ‘The Architecture of Happiness’, claims “the home is a place that succeeds in making more consistently available to its occupants the important truths that the wider world ignores”. The Interactive Wallpaper concept aimed to facilitate and develop this human need by allowing occupants to project surfaces with media that can remind them of their history and ideals, things which can easily be lost in the midst of their daily activities. The technical aspects of this project are illustrated on the following two pages.

IDEATION

EXPERIMENTS

INITIAL DRAWINGS VISUAL LINES IN ENVIRONMENTS A series of line drawings where I was experimenting with creating abstract graphic representation of physical spaces by making lines where I perceived paths within the environment. This technique is similar to the way Zaha Hadid designs many of her buildings.

GRAPHIC COLLAGES IMAGES OF URBAN ENVIRONMENTS These are a selection of graphic collages that I made whilst in Japan at teh beginning of my project. Using a photocopier to create layers though photocopying photocopies, the original images become unrecognisable as its data is over manipulated. Looking at graphic collage as a precursor to digital collage, this experiment focussed specificially on ideas of automated mashing of images, distortion and the idea of images only discernable to the individual for who it has personal significance. 60

TILE GRAPHICS One of several experiments into vector line graphics that could be tiled to form seamlessly flowing graphics on surfaces.

TILE GRAPHICS MAPPING INTERACTIVE TERRAINS Using these tile graphics as the grid to map surfaces in the interactive terrain, this methodology determines and articulates the boundaries of the feedback zones.

PHYSICAL SIMULATION TO DETERMINE DIGITAL AESTHETIC A simple example of some tests I did to simulate an effect I wanted to create digitally. Using a simple piece of cartridge paper on a screen, this test looked at ideas of blurring and focussing images within an interactive field. Sometimes it is easier to play with hands-on materials to test ideas than try to make them digitally.

PORTABLE INTERACTIVE DISPLAY Using the Wii Remote whiteboard techniques, coupled with a digital micro projector, I was able to create a highly portable interactive display. With the ability to simply pick up the device and project onto a range of surfaces, it is easy to use the Wii Remote and IR pen to transform any projected surface into a touch screen. This allowed me to test concepts without having to move cumbersome equipment.

QUICK 4 POINT CALIBRATION If the scope of the projected canvas is moved or the focus changes or anything is altered, then the 30 second manual 4 point calibration technique is nesssary. A similiar technique could be used to manually set the projection keystone digitally., allowing the user to quickly draw out the required perspective angle.

IR PENS In the past I have looked at infrared thimbles to generate touch screen feedback. This time I have opted for the more reliable pen model. I would prefer to eventually develop gesture-based feedback rather than point and click the IR pen to make the system more intuitive and user-friendly. 70

INTERACTIVE TILE Recreating the interactive tile simulation I made in my previous studio, I this time projected it with the micro projector. This allowed me to practically experiment with the concept as an individual tile rather than a collage of many and also to play with scale and surfaces. 72

DRAWING WITH LIGHT This installation involved a Wacom tablet and projector and experimented with digitally adding content to environments without physically manipulating them. The technique requires you to look at the projection surface rather than the tablet of the computer screen. Although Sweatshoppeâ&#x20AC;&#x2122;s video painting makes the tablet redundant, both techniques allow you to digitally draw - or in their case paint - directly onto the projected surface.

PROJECTION MAPPING Some of my experiments with mapping and then masking 3D surfaces with projection graphics.

SCREENS FROM PLANES IN THE PROJECTION SPACE What if the projection device was able to scan the 3D surfaces in its projection area and then automatically compile a collection of different sections from the different planes in the space? These sections would then form networked set of displays each being an independent screen zone.

INTERACTIVE SCREENS FROM PLANES IN THE PROJECTION SPACE What if these zones were then to become interactive with users manipulating their digital contents through gesture or touch? For example, each zone could show a different image form the users library of personal cultural signifiers. The user could navigate through the media as if flicking through a slideshow to compose their desire configuration for the projection collage.

PROPOSED OUTCOME

PROPOSED OUTCOME OVERVIEW

PROPOSED OUTCOME CONCEPT

As illustrated, this project attempts to harness our short attention span for visual culture using a system that reintroduces connections to histories and artifacts. I propose to create an intuitive system that helps the user prioritize their digital objects and allows them to create more in-depth relationshipâ&#x20AC;&#x2122;s to these digital artifacts. This new system will transcending the glancebased relationship we currently have with digital objects, permitting us to explore them in a more meaningful and intuitive way. More than just a digital database, this system will create an intimate, curated collection. The outcome of this project will be the creation of an interactive system that utilises visual projection to embed a virtual layer onto physical space. This system will focus on sight and touch as the core feedback modalities. At this stage of development, it is envisaged that this system will incorporate gesturebased interaction using Processing software Arduino hardware and modified Microsoft Kinect sensors to facilitate a live interaction within an automated projection mapping zone.

PROPOSED OUTCOME SCENARIO AUTO COMPOSITION

INSTALLATION

The projection device then assigns a media file from the userâ&#x20AC;&#x2122;s library to each of the zones.

The projection device is installed in the space and angled to the desired surfaces of the room.

USER MANIPULATION

TERRAIN MAPPING The projection device then scans the surfaces it will project onto and automatically establishes a composition of zones to reflect the different planes of the surfaces.

If the user wants to change the content that the device has automatically compiled, the user guestures in the area of the zone the user wants to manipulate and the system will suggest another file for that zone.

SECURED COMPOSITION Once the user is happy with the composition of the projection space the user can lock the composition and use the cultural signifiers displayed by the device as memory triggers.

DEVELOPMENT

TIMELINE APRIL

MAY

Determine Project Direction

Re-Define Project Direction

Initial Research Cultural Studies Scientific Theory Interaction Design Design Precedents Technology

Continued Research Cultural Studies Scientific Theory Interaction Design Design Precedents Technology

Early Ideation

Secondary Ideation

Start DVR

Technical Experiments

OCTOBER Project Reflection Project Outcome Completed DVR Document Printed Exhibition

Compiling DVR

JUNE

JULY

AUGUST

SEPTEMBER

Devolopemental Research

Materialisation of Project Outcome

Final Project Outcome

Intial DVR Chapters Finalised

Research, Ideation, Project Outcome DVR Chapters Finalised

Proposed Project Outcome Physical installation-based Experiments & Simulations Arduino and Processing Software Exploration

DVR Document Finalised Prototype Final Testing

Finish Physical Installationbased Experiments & Simulations Project Outcome Clarified Physical Prototyping Prototype Testing

BIBLIOGRAPHY

IMAGES REFERENCES 1

IMAGES REFERENCES 2

NOTES 1