The Spoken Word in Typography

Master’s Thesis by Ondrej Jelinek

THE SPOKEN WORD IN TYPO– GRAPHY

THE SPOKEN WORD IN TYPOGR APHY

Master’s Thesis

Ondrej Jelinek Master of Arts in Visual Communication and Iconic Research

Mentors

Prof. Michael Renner Paloma López Grüninger, PhD Jiri Oplatek University of Applied Sciences and Arts Northwestern Switzerland Academy of Arts and Design HGK FHNW Visual Communication Institute The Basel School of Design © September 2013

Abstract

This master’s thesis project examines the possibilities of visualizing non-verbal aspects of spoken language in written and printed form.

The opening part deals with chosen semiotic and linguistic theories and their relationship to spoken and written language. Afterwards practical examples from the field of concrete poetry and word-sound compositions are analyzed. On the basis of this inquiry, it is possible to point out essential differences between spoken and written language and address these issues visually. The practical part then approaches the question of how the invisible concepts related to our speech can be depicted, by experimenting with various typographical and letterform adjustments. Conducted experiments reflect the knowledge gained through the theoretical part, providing a fertile platform for a discussion.

Acknowledgment

I would like to acknowledge the guidance and supervision of my mentors – Prof. Michael Renner, Paloma López Grüninger, PhD and Jiri Oplatek. I would like to thank them for helpful suggestions and their willingness to meet me besides our scheduled meetings.

I would also like to thank my parents, family and friends for their endless support. Moreover, I would like to express gratitude to my partner, Adéla, for her love and support she has shown during my studies. Last but not least, I would like to thank all the teachers, assistants and classmates from the Visual Communication Institute for useful advices and a friendly atmosphere. Thank you.

9

INTRODUCTION

31

THE RHETORIC OF TYPOGRAPHY

73

CONCLUSION

TABLE OF CONTENTS

15

21

BETWEEN SEMIOTICS

NON-VERBAL

AND LANDGUAGE

COMMUNICATION

39

55

ON THE RELATIONSHIP

APPROACHES TO CONCRETE

LANGUAGE AND

THE DESIGN PROCESS

POETRY AND WORD-SOUND COMPOSITIONS

81

BIBLIOGRAPHY

89

THE PROCESS DOCUMENTATION

The Process Documentation

INTRODUC TION

9

The Process Documentation

10

Introduction

Human speech as an articulated representation in written form is a unique example of bridging between two signifying systems.

Even though they are tightly connected, there is one main distinction between them. This is the aspect of temporality that determines the benefits and drawbacks of their usage in specific situations. Whilst written text stays the same over time, articulated utterance is fixed to the moment, to when it is said. Spoken language is therefore useful for immediate interpersonal communication, without the demands of knowing its written equivalent.

Written language, on the other hand, enables us to disseminate knowledge more effectively and without the physical presence of an author.

My hypothesis is based on the idea that it is possible to create a typographic adjustment that would specify the way we read a word. On the other hand, some of the solutions could extend the possibilities of an oral performance by showing a word as an ambiguous and ever-changing entity. Consequently, a deeper understanding of a word should be established, even when no other context is given. Although the sound-image is a mental process of subvocalization, this is a question of whether it can be also depicted.

Because there is a certain gap between spoken and written language, this master’s thesis project examines the possibilities of visualizing non-verbal aspects of spoken language in a written and printed form. The main focus is not on the meaning of a word, but on other matters related to the process of articulation.

Showing speech as a complex and organic phenomenon is another goal of this project. The effect of the ‘visual presence of sound’ should enable the reader to appreciate a diversity of individual vocal renderings.

Visual experimentation in concrete and sound poetry served as a starting point for this research, as it provided a domain where the outcomes of my experimentation could be applied. Many artists in the past addressed the possibilities of the transcription of human speech, since the act of performance is crucial in this context. The most revolutionary contributions were made at the beginning of the 20th century, therefore my project can be regarded as a reaction to these efforts with more than a hundred years in between.

The main inspiration for this master’s thesis project emerged from the frequent use of glyph alternates that are a usual com-

11

Introduction

ponent of current fonts. I wondered if it would be possible to develop a form of system that would use these alternates to accentuate some of their acoustic properties, instead of using them for aesthetic reasons. If it were possible to create such a system, it would enable its users to write using an extended alphabet directly in the computer. However, creating a functional font is not a primary goal of this project.

The theoretical part of this master’s thesis project is devoted to language as a semiotic mode and to a systematic analysis of different approaches to concrete poetry and word-sound compositions. The knowledge gained from this theoretical research should help me not only to become orientated in this complex field, but should also provide the necessary vocabulary to describe and evaluate my own process.

The first part of this thesis looks at fundamental semiotic theories in the context of language. Because language is merely a system of signs, semiotic theories provide a basis for my research.

Using the concepts of traditional semioticians, the relationship between written and spoken language is scrutinized. The terms langue (language) and parole (speech), introduced by Ferdinand de Saussure, create a starting point for a critical evaluation of their relationship.

After looking at semiotic theories, language and non-verbal communication constitutes the theme for the following chapter.

Particular focus is placed on the relationship between written and spoken language and prosody (pitch, stress and rhythm); additionally facial expressions and gestures are taken into account.

By analyzing these features the basis for further exploration is established.

The next part is devoted to the rhetorical aspects of typography, exploring the influence of the form on the perception and meaning of a word. In this context, the work by Theo van Leeuwen helped me to get an overview of possible type-image relationships. The research in typeface personality was focused on the moods and atmosphere that can be achieved by various typefaces. Usually we are able to ascribe them to certain adjectives such as aggressive or feminine, but also neutral looking typefaces represent certain ideas.1 Regarding voice representation, typeface itself can also signify various vocal qualities.

Following on from this, selected examples are analyzed in order to provide an overview of practical approaches to concrete

12

Introduction

poetry and word-sound compositions and are divided into three groups. The first category is interested in the composition and layout of typography as a new semantic mode, analyzing experiments by early concrete poets. Expressiveness is the topic of the second group, looking at examples, where figural depiction is replaced by free and often subjective representation of words and sounds. Coded systems is the topic of the last group, demonstrating the most systematic approach to the representation of prosody.

The analysis of selected examples should help in evaluating my own experiments from diverse viewpoints.

The practical part of this master’s thesis is based on typographical experiments, trying to represent the spoken word and its qualities. By using a variety of different tools and techniques, it should be possible to find solutions which best fit the representation of sound and its qualities. During the design process sketchbooks were also employed to explore typographic qualities in a tool – and material – independent way. The image creation is based on a loose experimental approach in order to obtain a wide array of solutions addressing this issue and provide enough material for an objective comparison.

13

The Process Documentation

ON THE RELATIONSHIP BETWEEN SEMIOTICS AND LANGUAGE

15

The Process Documentation

16

On the Relationship between Semiotics and Language

In our everyday life we are constantly exposed to many different signs that help us to understand the world we inhabit. A sign can be described as “anything that communicates meaning beyond itself ”2, so any kind of communication is composed of various signs.

Basically, anything from a footprint to a complicated metaphor can be regarded as a sign, implying its heterogeneous character.

Because each communication is merely a system of signs, their importance in the context of my thesis cannot be underestimated. Semiotics is a scientific field encompassing different kind of signs, their meaning, structure and relation to their interpreters.

The main focus in the broad field of semiotics – especially on the relationship between the written word, sound and image – is given by the theme of this thesis.

There are two main semiotic theories – dyadic, developed

by Swiss linguist Ferdinand de Saussure, and triadic, constituted by American philosopher Charles Sanders Peirce. The main distinction in their approach is that Saussure uses his theories on a linguistic sign, while Peirce works with any kind of a signification (including linguistic). Since my main interest is in a written word and its sound equivalent, Saussure’s structuralist approach provides a foundation stone for the linguistic part of this project. Chosen concepts by Roland Barthes, who applied semiotic theories to photographic and advertising images, are analyzed and discussed at the end of this chapter.

The book ‘General Course in Linguistic’, which was composed by Saussure’s students after his death, is a collection of his fundamental linguistic and semiotic lectures, proposing a structural approach to the study of language. Even now this book serves as a basis for modern semiotic theories, especially thanks to his theoretical division of linguistic sign into signifier (acoustic image) and signified (concept). Saussure defines the linguistic sign as a “two-sided psychological entity”, located only in our brain. The acoustic image (sound-image) can be understood as a mental vocalization or subvocalization, which is the act of reading aloud in our mind. This implies that before we are able to decode a concept of a word, it first has to be mentally articulated. “The psychological character of our sound-images becomes apparent when we observe our own speech. Without moving our lips or tongue, we can talk to ourselves or recite mentally a selection of verse. Because we regard the words of our language

17

On the Relationship between Semiotics and Language

as sound-images, we must avoid speaking of the “phonemes” that make up the words.”3 Therefore if we want to speak about phonemes as building blocks of words, the act of articulation has to be physical, not mental.

Another important point in the Saussure discussion is that signs are generally established arbitrarily. In other words, there is no clear connection between the idea of a dog and a real dog or the word we use to describe it. A sign is on one hand ruled by a system in which it operates, and on the other it is dependent on its distinction from other signs. To put it another way, although the meaning is arbitrary, it is defined in relation to other words – in short, what represents the idea of a ‘dog’ also differentiates it from a ‘cat’. The same arbitrariness applies to the actual pronunciation, because in a similar manner there is no direct relationship between the letter ‘a’ and the way it is articulated.

The fact that basically any concept could be described with any word is supported by the very existence of different languages.

Though the relationship between signs is arbitrary in its essence, over time it is regarded as natural in any given cultural or social group. Furthermore, Saussure discusses specific words, such as onomatopoeias and interjections, that could disprove his idea of sign arbitrariness, but, again, by using the example of different languages this argument becomes irrelevant. The aspect of materiality is largely underestimated in his discussion, since he argues that materiality has no effect on the meaning of a sign. This was also one of the reasons he was criticized, since his opponents argued that materiality of a sign generates certain connotations.4 The American philosopher Charles Sanders Peirce proposes a different model of signification that is applicable to any kind of sign. In contrast to Saussure, Peirce studies signs in a scientific, logical manner that should not be confused with ‘beliefs’.5 According to his triadic model, a sign can be divided into three levels – representamen, interpretant and object. Representamen “is something which stands to somebody for something else in some respect or capacity. It addresses somebody, that is, creates in the mind of that person an equivalent sign, or perhaps a more developed sign.”6 In short, it is a representational form or object of a sign. Even though it does not have to be material, the most common representamen we encounter every day is the written word. Interpretant is not the receiver of information (person), but another sign that is created by representamen in the mind of

18

On the Relationship between Semiotics and Language

the receiver. It is the sense we get from a sign. The associations we generate when seeing signs are individual and in constant change, and are given by cultural and social circumstances. This is basically an extension of Saussure’s concept of a signifier. An object can be either a physical thing or a mental concept that a sign stands for. It is important to mention that both representamen and interpretant are connected to the same object. Peirce also introduces the term ‘semiosis’, which is an interaction between representamen, interpretant and object, or simply a process of meaning making.

Furthermore, Peirce provides sign typology according to the way signs represent their subjects, helping us to understand the signification of different kinds of images. Icons belong to the first category of signs, which are based on resemblance to their object.

For these signs not much additional knowledge is necessary since the appearance of a sign and the object to which it refers are similar and therefore familiar. Of course this definition is not fixed to images only, meaning that also resemblance in different qualities can be regarded as an icon. For instance onomatopoeias are lexical icons that try to resemble certain sounds by using linguistic devices. Indexical signs are not based on resemblance to their object, but rather on their existential or physical relationship. As in the case of a bullet hole, there is only a causal relationship between the bullet and the hole it makes. “In Roland Barthes’ words index points but does not tell.”7 The indexical nature of handwriting will be of particular interest when evaluating experiments conducted in the practical part. The concept of indexicality can be applied also to the production of human speech, caused by the physical act of air flowing through the speech organs. Symbols constitute a third category of signs, which are the most abstract ones and therefore linked to the perception of the interpretant.

These signs are established on cultural conventions and therefore must be learned. Symbols allow us to communicate on a highly sophisticated level, for example by using metaphors and diagrams.

On the other hand, Saussure argues that there is often a natural bond between a symbol and its meaning (the connection is therefore not entirely arbitrary), supported by the impossibility of changing one symbol for another.

The French philosopher Roland Barthes adopted the linguistic theories of Saussure and elaborated on them in the context of cultural studies. In his work ‘Image, Music, Text’, Barthes introduc-

19

On the Relationship between Semiotics and Language

es some crucial aspects regarding the way we generate meaning from images and words. In the chapter entitled ‘The Rhetoric of the Image’ he analyzes a famous Panzani advertisement by describing several semiotic layers on which we construct an interpretation.

Barthes differentiates signification on two levels – denotation (direct signification) and connotation (association), which is again largely given by cultural dissimilarities. Both denoted and connoted messages are received at the same time. The latter is regarded as an infinite chain of associations and usually has a stronger role in the final meaning making. It is almost impossible to have a ‘pure’ image without any connotative effects.8 Barthes also discusses the role of code in human communication, describing langue as a system or ‘le code de la langue’.9 Any kind of human communication is based on a certain structure – a code that is conventionalized by a society. Therefore it is necessary to be familiar with its structure to be able to decode it and understand the intended message. “Since the meaning of a sign depends on the code within which it is situated, codes provide a framework within which signs make sense.”10 Barthes uses an example of photography as a non-coded image (an indexical reflection of reality) and drawing as a coded image (a selective process, perspective, tool etc.). In the case of language, we may assume that it is composed of two different codes – a spoken and a written form. Codes help to orientate us in the complex world of signs, but they can also cause confusion. Diverse audiences can understand codes differently, which may lead to unexpected misunderstanding.

Furthermore, Barthes describes two possibilities for the relationship between word and image – anchorage and relay. Anchorage is the narrowing down of the possible explanations of an image by using captions. “...the text directs the reader through the signifieds of the image, causing him to avoid some and receive others.”11 On the other hand, there is relay, which is a complementary interaction of word and image, in a comic strip or in a diagram, for example.

In this way, Barthes neglects a differentiated approach to the possibilities of the interaction between word and image. A notion of this kind leads to the important question: Is it possible to anchor a linguistic message by visual means? This question brings us to the idea that we could use imagery in order to specify a linguistic message, which is exactly the case with typography.

20

The Process Documentation

LANGUAGE AND NON-VERBAL COMMUNICATION

21

Language and Non-verbal Communication

In general, language can be understood as a complex system of human communication containing both written and articulated expressions. In simple terminology it is “a system of distinct signs corresponding to distinct ideas.”12 The word ‘ideas’ used in this quote implies that language is largely culturally dependent, which means that every single language has distinctive structure and rules. According to Saussure, language is both a product of human speech and sum of all rules and conventions within which it is operating. The study of human languages is called linguistics and it can be further divided according to its main focus. The term parole introduced by Saussure is based on the idea that every single spoken utterance is highly individualistic. In simple terms, the way we say something is entirely. In contrast to the conventional written form, it is difficult to create an exact and detailed representation of particular speech. “The pronunciation of even the smallest word represents an infinite number of muscular movements that could be identified and put into graphic form only with great difficulty.”13 On the other hand, there is langue, which describes the inner structure and rules of our language and therefore is constant in time. Even though these rules might be changed over time, these alterations apply to the whole system of language and do not underlie individual execution of speech.

Non-verbal communication usually provides certain hints, making exchange of information much easier. Most of these aspects can be observed especially in interpersonal spoken communication, while some of them can be present also in the written form (such as typographical emphasis). For example, facial expressions and gestures can charge the message with additional meaning, stress or emotions, but there are several other factors that come into play.

23

Language and Non-verbal Communication

Spoken Language and Its Written Equivalent The process of articulation comprises various psychological and physiological activities. In the context of the sound production of human speech we usually speak about two main subfields – phonology and phonetics. Phonology is a subfield of linguistics interested in the cognitive articulation of the smallest units of language – phonemes. In contrast to this, phonetics is a field concerned with the actual physical production of language and its perception. According to Saussure the distinction between these two fields is as follows: “Phonetics is a historical science; it analyses events and changes, and moves through time. Phonology is outside time, for the articulatory mechanism never changes.”14 Saussure argues that written representation of language usurps the main role, although in reality spoken language is the essential component.15 Sometimes even if we just hear a word we can imagine the appearance of its written equivalent, which is given by our constant exposure to the written form. The written word has the advantage of relative stability over time, while spoken language is in constant change, which is also why it is a main domain of linguistics. The written equivalent of language is not limited by spatial and temporal character, therefore it allows us to spread knowledge effectively and without any time constraints.

Ultimately writing allows us to transmit extensive thoughts that can exceed the capabilities of human memory.

Writing systems can be broken down into three main categories – logographic, syllabic and alphabetical.16 There is one interesting aspect worth mentioning when speaking about the most widespread alphabetical writing system – in its essence it is deceptive. The written transcription is simply not faithful to what we pronounce, it is merely a conventionalized visual representation. For example morphemes, which are the smallest meaningful units of human speech, can have one fixed spelling but multiple pronunciations depending on the context.17

24

Language and Non-verbal Communication

The fact that the written form does not represent spoken language in a precise manner has caused an increased interest in an alternative way of transcribing it throughout the history. An important moment came at the end of the 19th century when French and British linguists created a universal form of sound notation commonly referred to as the International Phonetic Alphabet (IPA). Since the glyphs they developed directly represents sounds, this framework can be used to represent basically any language. This system also includes various prosodic indicators such as lexical stress, tone and length of articulation known as suprasegmentals (voice qualities extending over more sounds).

Although it might seem that the phonetic alphabet is closer to the actual production of speech, heterographs18 are an exception, where this kind of notation might cause misunderstanding.

According to Barthes’ terminology the IPA is a kind of subcode that has to be learned in order to understand it, which is also why it has been especially used by linguists, language students, teachers, actors and so on. Because IPA is composed of a large amount of complicated signs that differ from the glyphs we are used to, there have been several attempts to redesign it.

Especially in the 20th century designers like Kurt Schwitters, Jan Tschichold, Adrian Frutiger, Herbert Bayer and many others19 tried to come up with more intelligible systems that would be closer to the spoken language and at the same time easier to read for ordinary people (Fig. 1).

Fig. 1. Kurt Schwitters, Systemschrift, 1927

25

Language and Non-verbal Communication

Prosody Prosody is a part of non-verbal communication, which constitutes various features of human speech such as length, accent, stress, tone, intonation and others. These elements altogether indicate the form of a sentence (declarative, interrogative, imperative or exclamatory) and reflect diverse emotional states of the speaker.

In simple terms, it is not what we say, but how we say it, which in the case of sarcasm can reverse the literal meaning of an utterance.

The term prosody comes from the Greek word prosodía, described as “a musical term which appears to signify something like ‘a song sung to music’ or ‘sung accompaniment’, implying that prosody is the musical accompaniment to the words themselves.”20 Prosodic features have an important linguistic role in terms of meaning making, while features lacking this linguistic relevance are not classed as prosodic. In addition, paralinguistic features such as pauses and vocalization are not considered to be a part of prosody, but can have an important role in meaning making. A tension in our voice is a special case regarded as both a prosodic and paralinguistic feature.

Certain prosodic features can be described simply, but some of them are composed of many diverse aspects and therefore are not so easy to grasp. For example, while intonation can be easily defined as a pitch of the voice, rhythm is “is a complex feature involving accent, length, and tempo...”21 This unevenness causes further confusion since it is not exactly clear what can be regarded as a prosodic feature and therefore it is difficult to say how many features should be embraced by the term prosody.

Even though orthographic and typographic conventions include notation for some prosodic features such as punctuation and typographical emphasis, most of them are not present in the written form. This leads to a frequent misunderstanding, since prosody helps to narrow down possible explanations of information.

26

Language and Non-verbal Communication

Sound Symbolism and the Concept of Synesthesia In contrast to Saussure’s concept of sign arbitrariness, ideas of sound symbolism suggest that vocal sounds alone can have a specific meaning. It is possible to observe certain patterns when we analyze large amount of words containing similar letter combinations. For example words with the suffix ‘–ash’ or ‘–ack’ are often connected to destruction (crash, smack), while words beginning with ‘gl–’ can be associated with light (glitter, glow).

The contribution of neuroscience has to be considered in this context, because of its interest in synesthesia – transferring stimuli from one sense to another. It is an involuntary ability to convert impulses between two separate senses – such as some people are able to see colors when listening to music. In 2001 Vilayanur S. Ramachandran conducted an experiment with a focus group in which he asked the participants to connect two nonsense words ‘bouba’ and ‘kiki’ to two shapes – one sharp and spiky, the other soft and organic. “Now the amazing thing is 98% of people will pick the shattered piece of glass with the jagged edges, and say ‘oh that’s a kiki’, and the undulating amoeboid shape, ‘oh that’s a bouba’, even though they had never seen the shape before.”22 There seems to be a strong consistency in ascribing abstract shapes to sounds, which would imply that there is a certain relationship between the meaning of words and the way they are pronounced. This phenomenon is limited not only to the sound, but also to the appearance of a letterform. If we compare the angular shapes of the letter ‘k’ and its pronunciation with the rather organic form of letter ‘b’ and the way it is pronounced, we can assume that the design of letters also goes hand in hand with their articulation. Of course, in different settings the pronunciation of letters is changed, but it is possible to find situations where certain letter combinations represent specific ideas or – according to their pronunciation – evoke diverse emotions. For instance letters ‘g’, ‘k’ and ‘r’ when used in a word produce coarse sounds and are usually connected to negative concepts, while the

27

Language and Non-verbal Communication

pronunciation of letters ‘b’, ‘l’ and ‘m’ is much more softer. This can be seen for example in fantasy naming where positive characters have soft and friendly names (Bilbo), while negative characters are ascribed harsh sounding names (Sauron). In a similar fashion, somebody who does not understand either German or French, can only compare these two languages according to their sound.

No matter what the content is, this person might then assume that people speaking German are talking about negative things, while perception of the French language would be connected to more positive concepts. This is again based on frequent letter combinations and their conventional pronunciation.

28

Language and Non-verbal Communication

Facial Expressions and Gestures Interpersonal communication is always accompanied by certain non-verbal visual signals that help to support the message. These signals are stimulated by our nervous system and therefore occur unknowingly and sometimes also unintentionally. On the other hand, we are able to use facial expressions intentionally to emphasize what we are saying or to reflect how we feel at the particular moment. Whether we are happy or sad, confident or nervous, our appearance alongside the way we behave and speak, create a comprehensive image of our current condition.

This kind of visual stimuli can alter the meaning of words we say. The most important physiological parts contributing to this fact are definitely the eyes and the mouth, while eyebrows render another important factor helping to concretize the character of an expression. It allows us to distinguish, for instance, an evil smile from a friendly one.

In modern (mainly electronic) written communication, we can observe the frequent use of emoticons23 that serve as a substitute for facial expressions. Two dots and a curved line is enough to express our feelings through a simplified icon of a face, which provides a specification of the linguistic message that might otherwise be ambiguous or unclear. Broadly speaking, emoticons represent the tone of the voice and our emotions, transposed into a written form.

Hand gestures represent another non-verbal hint giving additional emphasis to what we say. Apart from the gestures that have a meaning of their own (such as the use of waving instead of saying “hello�), we use a large amount of hand gestures to accompany our speech. The main reason, when moving our hands while speaking is to stress certain words or to visualize what we are speaking about.

Sometimes it is much easier to show what we want to say than to actually say it, especially in the case of foreign languages. In contrast to facial expressions, which are operated by our subconscious hand gestures are strongly culturally coded and therefore have to be adopted. In a sign language various gestures are used for marking intonations, stress and other prosodic features.

29

Language and Non-verbal Communication

Many artists in the past were aware of the importance of facial and gestural expressions in the context of visual communication.

One such example can be seen in the work of French graphic designer Robert Massin, who is known mainly for his groundbreaking typographic compositions, where the typography becomes an organic extension of the human body. In his designs he combines distorted (stretched, bent or oversized) typography with illustrations of the human face or body in different expressive positions. To indicate who is saying what in more complex situations, various angles of baselines are used to connect a particular sentence with a speaker’s head (Fig. 2). A similar approach has a prominent role in Alexander Rodchenko’s propaganda posters, where it can be perceived that a person is shouting. This effect is even intensified by letterforms gradually growing in scale, which can be related to our conception of approaching sound waves (Fig. 3).

Fig. 2. Robert Massin, From Eugene Ionesco‘s La Cantatrice Chauve (The Bald Soprano), 1964

Fig. 3. Alexander Rodchenko, Poster for Soviet Union propaganda, 1924

30

Language and non-verbal communication

THE RHETORIC OF TYPOGR APHY

31

The Rhetoric of Typography

Although the main reason for written text is to transmit information, its appearance is (often unknowingly) experienced first.

When we look at a piece of typography, first we experience it visually. The overall layout is recognized instantly, whereas the recognition of a typeface comes into play later. Saussure claims that the way words are depicted (typeface, materiality etc.) does not affect their signification, but succeeding researches disproved this, stating that the ‘visuality’ of a word has certain connotative capabilities.24 Bellantoni and Woolman developed a terminology differentiating word image (an idea or concept) and a typographic image (visual impression), implying that there are two separate levels on which we perceive a word.25 The field of rhetoric mainly focuses on the available means of persuasion, in both the cases of speech and visual communication. “All communication is rhetorical: the meaning of an idea can’t be separated from the manner in which it is expressed.”26 This implies that even in the case of written communication, the visual aspect has to be considered. The visual impression of a written word enables us to communicate different qualities apart from its lexical meaning by concretizing or extending its possible explanations. Branding strategies often employ specific typefaces and type treatments only to evoke desired feelings or to tell stories.

These choices help to persuade customers to choose one product and not another.

Classical rhetoric describes three modes of appeal to persuade the audience – ethos (ethical appeal), pathos (emotional appeal) and logos (rational appeal). Relating rhetoric to the field of graphic design “…designer’s mode of appeal is expressed through choice of words, images, format, style, color, type, and materials.”27 From the classical point of view, rhetorical figures make the message more effective by departing from the ordinary use of speech. No matter in which field, the communication is effective if the message succeeds in fitting the specific occasion and audience. Rhetorical figures change the literal or usual meaning of an expression, falling into two categories – schemes (altering the order of elements) and tropes (altering their reference). In typographic design it is possible to utilize both of them to extend, specify or alter the meaning of a word. One of the most widespread tropes in the context of typography is probably the use of metaphor, employing materiality to “add a secondary reference to traditional letterforms.”28

33

The Rhetoric of Typography

Typography as a Semiotic Mode Until recent years, the semiotic mode of typography had been largely underestimated. Linguists have not addressed the issue of typography at all, since they regarded written language as a mere transcription of a – more important – spoken language.

Typographic research was focused more on the issues of legibility and reading processes than on hidden meanings connected to written text. However, in a world where computers are part of daily routine and where each of us is exposed to large amounts of advertisements, written language has become more visible than ever before. Especially in the field of advertising, it is mainly a question of balance between linguistic and pictorial means.

“Much of the cohesive work that used to be done by language is now realized, not through linguistic resources, but through layout, color and typography.”29 Nowadays anything that can be expressed through a visual form is much easier for consumers to ‘digest’. In short, what can be shown does not have to be said, or, rather, should not be said. Close collaboration between design agencies and neuroscientists has had the effect of continuing to improve strategies of effectively addressing their target audience.

Theo van Leuween in his article ‘Towards a semiotics of typography’ uses Haliday’s metafunctional theory, stating that both spoken and written communication simultaneously represents three ‘metafunctions’ – ideational (constructing representation), interpersonal (constituting social interactions) and textual (composing individual representations into coherent texts). Typography can be used ideationally to represent various actions and qualities, such as the word “spicy” composed of chili peppers. Interpersonal meaning can be achieved by using different typefaces to perform the text in a particular way, to express attitudes or feelings or by using punctuation marks to create emoticons. Punctuation marks themselves are an example of the textual metafunction, which is based on a thematic structure, composition, framing and salience. Multimodality of typography is another aspect brought up in this context. Whether it is colour, texture, three-dimensionality or movement, typography

34

The Rhetoric of Typography

is often interconnected with other semiotic means of expression to create a stronger and homogenous message. Regarding the representation of spoken language, kinetic typography can be considered as an interpersonal mode of communication able to visually translate speech, intonation and rhythm.

Signification of typography can be, according to van Leuween and Kress, described by two principles – connotation and experiential metaphor. Connotation means ‘importing’ signs from different context (historical, cultural or social), generating a new array of meanings. On the other hand, Johnson and Lakoff who developed the idea of experiential metaphor state that “the essence of metaphor is understanding and experiencing one kind of thing in terms of another.”30 This implies that the way we experience any thinkable phenomena can be freely transferred into a different context.

In addition, Mark Johnson in his book The Meaning of the Body elaborates on the importance of our bodily experience in the process of meaning making. The fact that we are complex organisms living in a certain environment implies that the way we make sense of something is based on ‘organism-environment interactions’. At the same time it also limits our thinking, as we are unable to think outside of it. The concept of image schemas is introduced here to describe “basic structures of sensorimotor experience by which we encounter a world that we can understand and act within.”31 The image schemas are stored in our neural maps and called upon when something we have experienced before occurs, assuming that they represent certain patterns we know from our environment. Taken in the context of this thesis, this means that we establish our understanding of certain typography on what we have seen or experienced before. Following this line of thought, sloppy handwriting, thanks to its irregularity and imperfection, instantly suggests either a children’s untrained hand or an attempt on a rebellion against typographic traditions.

Furthermore, van Leuween specifies the definition of experiential metaphor by employing the concept of learning by doing. “…a material signifier has a meaning potential that derives from our physical experience of it, from what it is we do when we articulate it, and from our ability to extend our practical, physical experience metaphorically, to turn action into knowledge.”32 Using the concepts of connotation and experiential metaphor it is possible to analyze typographic work from two different view-

35

The Rhetoric of Typography

points – historically, culturally or socially encoded associations, and intuitive, experience-based associations.

Following Piercean tradition, both speech and typography can be regarded as indexical, because of their close relation to the human body. Human speech is produced by air flowing through our vocal apparatus. As this is the effect of contracting muscles, there is a causal relationship between our body and the production of sound. On the other hand, typography can be understood as an index thanks to its tight connection to handwriting and the means of its production (tool and materiality). Even modern humanist typefaces are based on Renaissance manuscripts (in particular the Carolingian miniscule) and therefore contain shapes that are derived from the movement of the hand. This ‘touch of the hand’ is one of the reasons, why these sorts of typefaces have more personal attributes than others. As a matter of fact, humanist typeface is in its true sense an iconic representation of handwriting, which then, in the mind of a viewer, becomes an index for the movement of the hand.

36

The Rhetoric of Typography

The Personality of a Typeface It has been known that the choice of a typeface can alter the ‘tone’ of a printed document, but recent research in this field showed that typefaces also possess a certain personality. Apart from human-like characteristics, they can also evoke a mood or an atmosphere. “Every typeface has a different personality, and the ability to convey different feelings and moods...[Typefaces] can evoke strength, elegance, agitation, silliness, friendliness, scariness, and other moods.”33 Research papers by Eva Brumberger and Jo Mackiewicz served as the main source for my research in the field of typeface persona.34 Both of them have a specific approach to this theme. Eva Brumberger focuses more on general questions of typeface appropriateness and text comprehension by conducting quantitative research with several focus groups. On the other hand, Jo Mackiewicz tries to objectively analyze exact anatomic features that give type its qualities, stating that to simulate a particular mood or tone, the printed letter must follow specific formal elements. For example, a friendly-looking letterform should contain at least one of these characteristics: broken (imperfect) construction, rounded terminals, uppercase letters that dip below the baseline, a lowercase letter ‘e’ with oblique crossbar and a single-storey ‘a’ and ‘g’.35 Eva Brumberger is also interested in typeface appropriateness, evaluating different typefaces for three types of text – professional, violent and friendly. In general, people are not very influenced by a typeface when reading entertaining texts. In contrast to this, if a text contains conflict or tension, recognition of appropriate typeface is increased. Broadly speaking, the more qualities shared among the content of a text and used typeface, the better its overall impact. Furthermore, readers make these judgments based on their experience, on what they have seen before. For instance, gothic or blackletter typefaces have, due to its prominent use by the Nazi party after the year 1941, 36 strong ideological connotations (they could be a cue for a German accent). However our bodily experience tells us that, in the context of vocal qualities, it

37

The Rhetoric of Typography

could be regarded as aggressive. This kind of impression is evoked by its formal qualities – pointy and spiky letterforms may suggest direct physical harm. Naturally, this experience changes in time, therefore also the perception of a particular typeface is formed according to what we have seen before and what we remember.

Eva Brumberger concludes that selection of typeface does not have a big impact on the meaning of the text, implying that even if the text and used typeface indicate exact opposites, the meaning remains unchanged. This is particularly observable in the case of words with strong, emotive meaning because these words already have powerful associations, which are not easily altered.

Typefaces, thanks to their rhetorical abilities, are commonly used to evoke a particular atmosphere. There are of course certain qualities that can be distinguished as indicators for various vocal features; for example, Comic Sans can be regarded as a playful and friendly voice, while Helvetica can be understood as a neutral and cold tone of voice. In a similar manner feminine looking typeface (such as Edwardian Script) can indicate a high-pitched female voice, because it reflects the subtlety of femininity. In this context, it is rather difficult to manifest these qualities by only using a particular kind of typeface, since we are used to reading them every day and therefore read them normally. Rendering such qualities in an oral performance would be merely a subjective interpretation of a typeface, while for untrained people it is often hard to distinguish typefaces at all. It is a question of how a reader can be lead to perceive different typefaces as indicators for vocal changes. This awareness could probably be achieved by using unusual or experimental typefaces that would challenge our reading habits.

38

The Process Documentation

APPROACHES TO CONCRETE POETRY AND WORD-SOUND COMPOSITIONS

39

Approaches to Concrete Poetry and Word-sound Compositions

After studying theoretical approaches to the relationship of word and sound, in this section practical examples are analyzed and compared. Evaluating the qualities in selected works of inspiring artists and designers should enable me to observe the crucial moments and goals of their endeavor. Before the analysis began, certain categories had to be defined in order to provide a more systematical overview. As is known from art history, it is always hard to strictly set the boundaries between different styles and movements. This is also why defined groups should not be understood too rigorously. The following categories were developed with a focus on form, content and purpose rather than an exhaustive historical description. Naturally, some of the listed examples could also fit into other categories, but their prevailing qualities determined that they be mentioned within a particular group.

Early examples of a ‘shaped’ or ‘pattern’ poetry survived from the times of Ancient Greece, where the poem was attached to architectural structures and therefore the lines of text had to be adjusted accordingly. Another reason for this kind of solution was also the supplementary communicative value – the visual form transmitting content of the poem at first sight. This can be regarded as the predecessor of what is referred to as concrete or visual poetry nowadays. One of the pioneers of visual poetry, Guillaume Apollinaire, in his calligrammes37 utilizes a typographic form as a device to reflect the subject of a text. Such as in the poem Il Pleut (It’s raining), where lines of the text are composed in vertical lines, so that they resemble five streams of pouring water.

Concrete poetry is based on the idea of double signification, where the iconic signification and the linguistic message are delivered simultaneously. The visual impression is immediate, while the linguistic content has to be read and therefore takes longer before it is processed. “The linguistic sign, which constitutes a complete system in itself, functions as the first term of the visual sign, which expands to encompass a second signified at the visual level. In this manner, the written word serves as a support for the visual message.”38 The way we read is, according to Saussure, based on our familiarity with a particular word: “We read in two ways: a new or unknown word is spelled out letter by letter; but a common, ordinary word is embraced by a single glance, independently of its letters, so that

41

Approaches to Concrete Poetry and Word-sound Compositions

the image of the whole word acquires an ideographic value.�39 This would imply that in reality we read the shape of the word, not each letter separately. A similar process is present also when looking at concrete poetry, where the poem shape emphasizes its ideographic quality. The whole field of concrete poetry can be regarded as a contribution to the eternal discussion on the similarities between poetry and painting.

42

Approaches to Concrete Poetry and Word-sound Compositions

Composition Probably the most widespread approach to concrete poetry is the one utilizing the layout of words in figurative or abstract compositional forms. The position of words on a format creates a new semantic level, implying that words are charged with an additional meaning apart from the lexical one. After the invention of the typewriter in 1860s it became a common tool for concrete poetry besides the traditional letterpress and handmade lettering. Since each letter is assigned the same sized space in a line it can be easily arranged both horizontally and vertically, while such a composition can be made much faster than using a letterpress. It was the U.S.-based Black Mountain School that made use of the tab key “to develop a set of easily reproducible visual cues intended to govern the transmediation between oral and written.”40 The poem Easter wings written by George Herbert in the 17th century can even be, from our perspective, still perceived as modern. Herbert introduced a new way of laying out the typography, so that the shape of the poem resembles birds’ wings. What is also noteworthy is its orientation – lines are composed vertically, in contrast to classicist typography which commonly employs centred aligned, horizontal text.

The idea of using white space to demonstrate Fig. 4. Stéphane Mallarmé, From Un Coup de Dés Jamais N’Abolira the interplay of style and content is present in Le Hasard (A throw of a dice), 1897 the work of the French symbolist poet Stéphane Mallarmé. His main contribution in this context can be especially seen in the poem Un Coup de Dés Jamais N’Abolira Le Hasard (A throw of a dice) where he works with concepts of hypertextuality and visual silence (Fig. 4). In this poem we can see, that the ordinary way of reading is broken, altering the layout on each page and thus changing the syntactic structure of otherwise linear language. In other words, spatial syntax replaced the common grammatical syntax here, giving the reader more and diverse options of reading and interpreting the text. The analogy to painting is clear from his idea of “a simultaneous vision of the Page”41, which suggests preferred overall perception of the

43

Approaches to Concrete Poetry and Word-sound Compositions

layout to the linear one. A similar approach to the page layout as a new semantic platform can be perceived in the work of E. E. Cummings, Augusto de Campos and many other concrete poets.

Inspired by a car accident, Italian futurist Filippo Tomasso Marinetti was fascinated by speed, movement, violence and the visual representation of these phenomena. A cover of the book Zang Tumb Tumb, which reflects his impressions of the battle of Adrianople, constitutes an early example of typographical synesthesia – using typographical alterations to represent sounds (Fig. 5.). Here, Marinetti introduced his concept of parole in libertà (words in liberty). The concept was based on the use of phonographs for voice recording, developed by Guillaume Apollinaire,42 which allowed him to set free words and represent onomatopoeias (words imitating sounds around us). Even though the main carrier of the additional information is the composition, different typefaces and type sizes are used to indicate the strength and character of the sound. For instance, the gradual enlargement of successive letters can be perceived as sound that was initially quiet, but the closer it is, the louder it gets. We are almost able to differentiate the speed and nature of the sounds such as the rhythmicity of vehicles cruising around the battlefield or the overwhelming noise of gunfire. Marinetti exploits the potential of page format by using it both as a picture plane and a graphic field.43 What is really striking and innovative in this composition is the varying angle of the baseline for each of the words. This heterogeneity in placement and rotation helps to create a notion of interweaving sounds that are not governed by any rules, making this composition dynamic and impactful.

“This new array of type, this original use of characters, enable me to increase many times the expressive power of words. (…)My reformed typesetting allows me to treat words like torpedoes and hurl them forth at all speeds: at the velocity of stars, clouds, aeroplanes, trains, waves, explosives, molecules, atoms.”44

44

Approaches to Concrete Poetry and Word-sound Compositions

Among the artists that work with an unusual placement of words and letters the name Ilja Zdanevich has to be mentioned. In his poem Easter Island, Zdanevich consciously dissolves traditional orthographic conventions by grouping letters in a way that there is no clear reading direction. This gives the reader the option to develop their own story, although none of the readings actually make sense. Another noteworthy artist is the co-founder of the De Stijl movement, Theo Van Doesburg, who utilizes various vertical alignments of typography to achieve the notion of rhythm.

Franz Mon is a representative of the post-war sound poetry, interested in sounds that are similar to an actual language, but are composed of nonsense words in an unusual layout.

Fig. 5. Filipo Tommaso Marinetti, Cover of Zang Tumb Tumb, 1914

45

Approaches to Concrete Poetry and Word-sound Compositions

Expression It is rather difficult to define the characteristics of this group, since each of the representatives has a highly individualistic approach to typographic solutions. Most examples from this category belong to the field of sound poetry, where articulation and sound attributes are the focus. “Ideally, the term ‘sound poetry’ would describe poetry which emphasized acoustical properties rather than the meaning of the words, i.e. the conventional relationship between sound and semantics in poetry would be reversed and the connotations would accentuate the poems sonic, rather than semantic qualities.”45 Because of this, sound poets often use nonsense words in order to detach sounds from the meaning, even though some of them work with onomatopoeias or even commonly used words. Such attempts can be regarded as a continuation of the work of the Futurists, Dadaists and other artists from the beginning of the 20th century, interested in trans-mediation between our senses. The main distinction is that (compositional) concrete poetry is usually written for a general audience, which is able to read and understand it without explanation or performance of the author. On the other hand, expressive sound poetry often utilizes a personalized way of notation for reading out loud, which is often hard to read and therefore even harder to interpret. As a result, only the author himself is able to perform his own poem in the right way, because somebody else might not understand his intentions. Johanna Drucker, in her essay Not Sound, asserts that “any form of represented text can be used as a score for oral performance”. Furthermore, she introduces the term graphical codes, which are separated from the lexical meaning, yet provide important hints on how to understand a text even before we start reading. The material properties of the graphic codes “instruct and provoke our reading of poetic works, whether they are notations for sound or not.”46 Another crucial aspect in this context is the materiality of a written text that helps to provoke certain assumptions or associations to possible readings of a text. Our understanding of materiality is largely based on the image schemas and the experiences we have gained throughout our life. Graphic variables such as size, shape, value,

46

Approaches to Concrete Poetry and Word-sound Compositions

colour, pattern, orientation and placement can be employed in creating a notation that resembles some of the acoustic properties.

The invention of the tape recorder in the fifties led to the real revolution in the field of sound poetry. Until then, sound poets were limited only to work with the visual representation of sounds and their interpretation in performance, based on the ideas of the Futurists and Dadaists. Afterwards a tape with recorded sound also became a medium for exploration and experimentation, which allowed the combining and overlying of different sound tracks as if working with a film tape.

One of the most innovative artists in the context of expressive notation for sound poetry is definitively Bob Cobbing. What is especially characteristic of his work is his fascination with performance and modes of its visual representation. He was interested not only in vocal performance, but also in gestural expression and even dance. Re-using was one of his main approaches to generating expressive visual ‘scores’ for a performance, especially by using cut-ups that were often processed several times. In his poem Shakespeare Kaku, Cobbing uses the overprinting of stencils to achieve the notion of movement (Fig. 6).

“The poem serves as a notation for vocal and gestural improvisation, written for three-voice and organ live performance.”47 It is composed of fifteen typographical assemblages that, for the first time in Cobbing’s work, utilize different compositional strategies giving the text further meaning. This spatial setup demonstrates how Cobbing utilizes a book as a medium to suggest different aspects of a performance. The ambiguity and unclearness of this poem give rise to many possible interpretations and indefinite ways of reading it. This implies high demands on the reader’s interpretative capabilities, in other words “much of the creative work must be done by the reader.”48 Also materiality plays an important role in this example because while the traditional letterpress technique ‘reserves’ a certain amount of space for each letter, here we are confronted with numerous overprints on a small space. Cobbing also worked with more experimental kinds of notations for his performances than just an ordinary printed or written text. Such as in his performance entitled Stone Tones, where the participants performed according to the shapes and structure of a stone they touched. “The rock acts as a three-dimensional graphic score. Irregular patterns upon the rock’s sur-

47

Approaches to Concrete Poetry and Word-sound Compositions

Fig. 6. Bob Cobbing, From Shakespear kaku, 1972

Fig. 7. Steve McCaffery, from CARNIVAL the second panel, 1970–75

48

Approaches to Concrete Poetry and Word-sound Compositions

face as well as the rock’s shape, weight and feel provide cues for vocal and gestural realizations.”49 Steve McCaffery is another artist whose poems also appear to be more visual compositions than scores for a performance. His early publication CARNIVAL the second panel (inspired by Ezra Pound’s Cantos) is characterized by wide use of different materials and tools (Fig. 7). The legibility is strongly reduced by an overprinting and smudging technique, supporting the pictorial layer of expression. At first the reader is confronted with the whole piece, distanced from the content and language, while later on McCaffery encourages the reader to “…move freely, as the language itself moves, along one and more of the countless reading paths available…”50 When speaking about expressive typography, gestural aspects of a handmade typography have to be considered as well.

One of those who explored the possibilities of hand movement in a production of sound poetry was the futurist painter Giacomo Balla. Mixing both uppercase letters and script, Balla created a very personal kind of sound notation, applied to nonsense words.

In a similar way in Russia, Velimir Khlebnikov utilized brush technique for the development of new words, which is commonly referred to as Zaum.51 His organic typography is often combined with non-figurative elements resembling waves or floral motives, and sometimes even with human figures and faces. Among other noteworthy artists working with expressive scripts at least Valeri Scherstjanoi and John Cage should be observed. For some sound poets even the gesture itself can be turned into a sound by amplifying their own muscle contractions.

There are, of course, more artists worth mentioning in the context of expressive word-sound compositions, but there is unfortunately not enough space to elaborate on all of them. For instance, Henri Chopin, Larry Wendt, Clive Fencott, Peter Finch and Spencer Selby can also be regarded as major contributors to this field. Each of them has a unique approach, ranging from simple handmade calligraphy to visually and technically complex multimedia solutions.

49

Approaches to Concrete Poetry and Word-sound Compositions

Coding After looking at more individualistic approaches to the expression of sound and speech, coded systems generally belong to the more self-explanatory and, thus, intuitive examples. These solutions are characterized by systematic and comprehensive structures that are developed in order to offer a more truthful transcription of spoken language. Most of them work with distinctively adjusted typography, while others use non-figurative elements, often inspired by musical notation, to create a specific kind of diagram.52 Even though most of these systems are intuitive, they still underlie Barthes’ definition of codes – they have to be learned in order to decode them. Following his line of thought, these codes create a structure in which each sign (in this case a graphical element) has a fixed meaning.

The first attempt at a more precise and systematic notation for vocal renderings was made by Joshua Steele, who in 1775 published An essay towards establishing the melody and measure of speech to be expressed and perpetuated by peculiar symbols. Steele states that there is a certain correlation between speech and music, utilizing adjusted musical notation to show five features of speech intonation – accent, emphasis, quantity, pause and force (intensity). His solution is based on the idea that the voice never stays in one pitch, ascribing each syllable a particular line – straight line, curve or circumflex. To these lines he then added a tail indicating the length of a syllable (Fig. 8).

Even though this system reflects the main idea of how something was said, it does not count with varying ranges of tonal spaces that differentiate speech from music. There are many artists that also use a musical score as a starting point for their wordsound experiments. For instance, the Fluxus artist John Cage or the Portuguese poet Fernando Aguiar, who uses a musical staff in a similar way to Steele – to show a scale that relates to the pitch of the voice. On the other hand, his approach is dissimilar since he uses the staff merely as a grid on which the letterforms are directly painted.

In the middle of the 20th century Ernst and Marion Robson introduced another system, which employs typography in a more prominent way. Their Prosodynic notation, produced by hand

50

Approaches to Concrete Poetry and Word-sound Compositions

with the aid of LeRoy templates53, indicates the duration (speed), intensity (power level), intonation (pitch) and vowel pitch modulation of pronunciation by adjusting individual letters (Fig. 9).

Three possible durations are based on stretching the letterforms horizontally, while three steps of intensity are shown by different boldness. Fundamental pitch is demonstrated by a shift of the baseline of the respective letters – lowest pitch by depressing them, middle pitch by their normal position on the line and highest pitch by elevating them above the baseline.

The cues for vowel pitch modulation are only two – for rising or falling intonation,instructing the reader to adjust an intonation of a vowel in controllable periods. Thanks to its wide scope, this system grants its user relative freedom by working with 46 prosodynic states per syllable in total. Because this system was so groundbreaking in the context of poetry and performance notation, compositions using prosodynic script were published in several avant-garde magazines. There are many benefits of the prosodynic print, such as the possibility to reflect author’s intention of speech or to visually clarify intonation patterns. The French Lettrist movement developed a specific kind of notation based on numbers, constituting a predecessor of this systematic approach. In this context, numbers were used to “indicate a particular vocal sound and had a sort of dictionary or vocabulary of sounds made by the voice.”54 Their poems also employed forms taken from musical notation. Their interest in combining writing with more visual means of communication led to the development of the term hypergraphy, which is based on the idea of transmitting information through a mixture of ideographic, lexical and phonetic means.

Fig. 8. Joshua Steele, Transcription of Pope’s Happiness, 1775

51

Approaches to Concrete Poetry and Word-sound Compositions

Fig. 9. Ernst and Marion Robson, Poem Crows in prosodynic script, 1946

FILTERKOMBINATION AUS DEM GESTALTUNGSINSTRUMENT

0.00 Spracheingabe und Zuweisung des Schriftschnittes

2.01 Lautstärke Schriftgrösse richtet sich nach der Lautstärke

2.03 Intensität Sprechintensität ergibt die Strichdicke

3.03 Artikulation Unartikulierte Buchstaben werden verstümmelt

3.04 Redefluss Gebunden sprechen verbindet die Buchstaben, abgehackt zerstückelt sie

4.03 Höhenkurve Die Tonhöhe generiert ihr Sprachgebirge

Fig. 10. Pierre di Sciullo, Le Quantage, 1989

52

Fig. 11. Meggi Zummstein, Visualisierung von Sprache (Visualization of Language), 2001

Approaches to Concrete Poetry and Word-sound Compositions

Among the contemporary designers that work with coded systems nowadays, the name of Pierre di Sciullo has to be mentioned.

His graphical approach results in a wide range of experimental phonetic alphabets, such as the Quantange where new glyphs are introduced, representing the pronunciation of a letter in a particular context (Fig. 10). In short, there are as many glyphs for each letter as there are ways of pronouncing it in French. Another recent contribution was made by Meggi Zumstein, who in her diploma thesis Visualization of Language (Visualisierung von Sprache) in 2001, developed a comprehensive system of filters, reflecting several features of human speech (Fig. 11). Certain similarities can be seen in relation to the notation systems developed by Ernest Robson or the Futurists and Dadaists, who also employed typographic and spatial alterations, such as letter size indicating loudness, space to indicate pauses, boldness for emphasis, shifted baseline to show intonation, etc. It is possible to observe some previously mentioned effects in an example of an optophonetic poem k’perioum by Raoul Hausmann, which is not entirely coded (different sizes of letters do not underlie any rules), but our interior ear can decode various sizes of letters as a cue for a varying intensity of voice (Fig. 12). In contrast to these solutions, Zumstein also employs several non-figurative elements that on one hand make the reading harder, but on the other strengthen the overall visual impact. To conclude this chapter, coded systems are also present in comic books, where adjusted typography in combination with various types of speech bubbles, indicates emotions, sounds or the voice of the speaker.

Fig. 12. Raoul Hausmann, kp’erioum, 1919

53

The Process Documentation

THE DESIGN PROCESS

The Design Process

After the fundamental research on the relationship of written and spoken language, crucial differences among them can be pointed out. The fact that something is missing in a written form provoked my interest in how this ‘gap’ can be shown on a visual level. At the very beginning of the design process, a large amount of expressive typography examples was collected, to get a basic idea of what can be achieved only by adjusted lettering. Based on this analysis I was able to define research questions and distinguish various approaches to address this relationship visually. In addition, some influential contributions were discovered during the design process, helping me to find the right direction for new experimentation and also giving me the opportunity to compare my solutions with other similar approaches to this theme.

To get an overview of possible visualizations of voice, I have used Voice Analyzer and Praat computer software. Especially for the coded systems this kind of initial analysis was necessary in order to work with exact data, while for the expressive solutions it served as an inspiration for possible ways of visualizing sound.

Research questions

What can be transmitted to the reader within a letterform?

Is it possible to visually represent the sound image of a word and thus narrow down its possible explanations?

Where is the boundary between phonetic and visual?

Can a form be articulated?

How can features of a human’s voice be graphically translated?

Which solutions are successful in terms of evoking the nature of a voice/sound and its ephemeral character? Which tools and techniques are effective?

How can the reader be instructed to follow various adjustments as a cue for vocal alterations?

57

The Design Process

Coding After the initial research into prosody, I decided to start off the experimenting phase by creating coded systems. Since language and typography are also part of a certain structure, it seemed logical to me to begin with the most systematic approach. In the first place it is important to consider which kinds of codes (demonstrating some prosodic features) are already conventionalized in typography. The most prominent example is the usage of punctuation marks and typographical emphasis. The problem with punctuation marks is that they are not very distinctive within a text block – we recognize the form of a sentence only after our eyes reach the last character. On the other hand, this attribute is very effective for longer texts, because it does not capture our attention too much and the text block remains uniform. The typographical emphasis is used to highlight some parts of a text, but it can also demonstrate certain vocal attributes. There are several methods of typographical emphasis – different cuts (italic, bold), capitalization, letter spacing, underlining and few others. Regarding the voice, italic cut is often used for quotations – giving the text a more personal character, since italic is closer to handwriting (thanks to its slant and partial connections). The bold cut of a typeface can be then ascribed to a shouting voice, which is also the case of a CAPITALIZED text.

After the initial research, important questions arose. What is it possible to show on one letter? How can the reader be instructed to read a word in a particular way? In the beginning the main focus was on showing intonation, intensity and length as the main prosodic features, creating the additional meaning.

The first phase was composed of two different approaches – designing new representational signs that would indicate one or more of the prosodic features, while the second one was based on combining common letterforms with simple geometric elements (diagrams). The initial experiment was focused on letter repetition and the proportions of letterforms. When stretched, letters can signify the length of pronunciation, while a recurring letter can also be used to indicate the pitch of our voice. Overlapping letters create a new glyph representing the letter and some of its sound properties, while reflection of letters can evoke the notion of an echo (Fig. 13).

58

The Design Process

Fig. 13

Fig. 14

Generally speaking, letterforms are composed of different strokes, while their thickness can be linear or various according to the tool used for their production. Apart from the materiality, the character of a line reflects an additional meaning derived from its historical and cultural connotations. But when it comes to more abstract and tool independent line qualities, our bodily experience also has to be considered. Since it is always easier to work with two extremes, circles and triangles were utilized as a code to show friendly and aggressive tone (Fig. 14). Our perception of such abstract elements can be described by using the concept of experiential metaphor, which in this case is given by our interaction with pointed and rounded objects in our world.

59

The Design Process

The diagrammatic solutions employ simple non-figurative graphical components that should also cue the reader for vocal alterations. In one experiment, a letter is divided into five units of the same size, evoking the image of a ruler (Fig. 15). Each of these five units can be then be highlighted to indicate the intensity of the voice. This simple arrangement of course might not work ideally in the case of more complex or rounded letterforms, where this scale would be distorted or too indistinctive in comparison to the letterform itself.

Another experiment works with straight or bent curves, indicating the type of intonation. By adding more lines, it is possible to demonstrate intensity as well, but as it showed later, boldness of characters is a more intuitive indicator for loudness. It emerged that when these lines are placed next to the letterforms and not over them, the overall intention is much clearer. Furthermore, by utilizing compositional forms that follow direction of the lines, this aspect is even more accentuated (Fig. 16).

The next concept employs vowels as the main carrier of prosodic information. Because vowels can be opened or closed based on the position of our jaw during articulation, letterforms are distorted accordingly. This means that, in particular instances, a vowel can be stretched vertically to indicate maximal openness of the mouth when pronouncing it (Fig. 17). An idea like this is based more on phonetics than the prosody itself, but it shows how our bodily experience can be connected to an appearance of a letterform. Another self-evident aspect is that the different quality and thickness of the stroke represents the volume of the voice.

Whether a vowel should be whispered (dotted) or shouted (thick) is apparent on first sight and, therefore, intuitive. At the same time, essential letter shapes were combined with simple curves, creating a glyph charged with additional information (Fig. 18).

In the next example, more codes are combined to create a complex diagram of human speech. Different kinds of underlining and framing were explored here as a code marking both lexical (word) and prosodic (sentence) stress. One of the ideas was to make an underline from recurring punctuation marks, which would indicate the form of the sentence in each part of the text. Various positions of letters on a format signify the pitch of the voice, while the size of the letters is differentiated to visually emphasize which of them are not pronounced that loud. Apart from the letter size, the proportion, and the position on the for-

60

The Design Process

Fig. 15

Fig. 16

Fig. 17

Fig. 18

61

The Design Process

mat, this system employs cues indicating the form of the sentence (question) and the primary and secondary lexical stress (Fig. 19).

When more codes are combined in a more complex diagram, the idea of playfulness can come into our mind. This example is somewhat exaggerated, because for normal speech the differences in stroke thickness would not be so striking. Even though this solution contains a large amount of prosodic information, it is not very successful in terms of transmitting acoustic qualities to the reader. The meaning mediated through a complicated system is less evident, rendering the reading speed naturally slower, since the letterforms become more complex. This process is similar to reading maps or diagrams – we have to know or learn something in order to understand all the codes employed.

Another experiment was based on the idea that lower case letters are percieved as not that ‘loud’ in compare to upper case letters. In this way, characters that should be articulated silently are written with lower case. Additionaly, letters are interconnected to indicate syllables, creating new phonograms for certain sounds. This is an example of a semi-coded approach, since some of the letterforms are distorted in an expressive way, but on the other hand demonstrate a character of intonation (Fig. 20).

62

The Design Process

Fig. 19

Fig. 20

63

The Design Process

Composition Any kind of written text is fundamentally composed of graphic elements. The only distinction from any other geometric form is that the shapes we recognize as letters have a certain meaning, which is conventionalized by our society. Nevertheless, any typography can be regarded as a graphic form, acting like any other visual element. Typographic solutions can have a strong pictorial effect by resembling forms we know from our environment, such as water waves or branches of a tree. Basically anything we have experienced in our everyday lives can be transposed into the world of typography. This implies that we experience typography in the same way as we experience our environment. In this sense, a text placed in the upper part of the paper seems to be lighter than a text placed on the bottom of the page.

Since there is a limited number of possible positions on the format, the variety of examples is relatively restricted. Using only one typeface in one size and one cut also means that the outcomes are not as visually impactful as other experiments. After analysing examples from the theoretical part, simple exercises were conducted altering the placement and rotation of words.

The concept of experiential metaphor can be used when speaking about typography as a representation of the voice.

Probably the most simple, but also the most effective, way of showing intonation is the position on the format and in relation to other words (letters). By altering the common placement of letters on a baseline, high and low pitch can be easily shown. This fact is connected to the physicality of our body, which helps us, thanks to its anatomy, to distinguish what is up and down, left and right. Again by using metaphor, higher frequency of the voice is connected to up, while lower frequency is regarded as being down. In this sense, a text set on various baselines may suggest changes in pitch modulation (Fig. 21).

Fig. 21

64

The Design Process

When words are arranged next to each other horizontally, but their baseline shift is altered, larger white space creates a gap in reading, which can be perceived either as a cue for a pause (similar to Mallarmé’s visual silence) or an alteration of the pitch. In some cases when words have a lot of white space around them, their meaning is changed, while their proximity to other words may generate new associations. A different angle of the baseline adds to visual dynamism, while the aspect of intonation is strengthened, because the intonation curve is continuous and thus easier to follow (Fig. 22). By employing different sizes of a typeface, the effect of distance is achieved. The composition alone is not capable of creating a sense of depth, since the concept of space is based on relative sizes. This again can be related to the perception of phenomena in our world. For instance, when somebody talks with a constant intensity and then begins to move away, their voice naturally gets more and more quiet. Therefore our notion of intensity is related to the size of words – bigger means louder, while smaller is quieter.

Fig. 22

Another compositional concept was developed, utilizing a book as a medium to show a dialogue (Fig. 23). To try out my ideas I chose a scene from the movie Pulp Fiction where Jules talks to Brett about Mr. Wallace, especially because it is charged with prosodic contrasts and emotions. The decision was made to work with an accordion book, which can be unfolded to see the whole dialogue at once – in a similar way to a timeline. The format is divided into two horizontal parts – the upper one is reserved for Jules, the lower part showing Brett’s line. This division corresponds to the actual situation in that scene, because Jules is standing above seated Brett. Thanks to this format it is possible to easily see the breaks which each particular

65

The Design Process

speaker makes – giving more emphasis to the words that are surrounded by a lot of white space. The visual silence around words here matches the prosodic emphasis, fulfilling the main intention. Apart from this, various shifts of a baseline could indicate the pitch of a voice. Finally, this concept allows the visualization of two people speaking at the same time, which is hard to indicate in common theatre or movie scripts. With this approach, performance notation for more participants can be created, showing the pauses and overlapping of individual lines in a lucid manner.

Fig. 23

66

The Design Process

Other exercises were based on the position of words in relation to each other. Ambiguous words were selected for this experiment in order to observe how their pronunciation and meaning change. The words were arranged into vertical columns, which were then shifted making the reading direction unclear, offering the reader more possibilities in which to approach it. The reader can move freely, modulating the intonation according to the position of the previous word (Fig. 24). Another example is questioning different pronunciation of the same letter combination.

When the words are arranged in a way that the corresponding letters are above each other, the intention is evident. This system therefore invites the reader to read the words in a new and creative way (Fig. 25).

THIS HELLO

THERE YES

THERE

YES

BLACK REALLY

NO

MAYBE

THERE YES

YES BLACK

HELLO MAYBE

HELLO REALLY

THIS BLACK

THERE HELLO

THERE YES

THIS MAYBE

THIS

BLACK MAYBE

HELLO MAYBE

BLACK NO

THIS REALLY

REALLY NO YES

HELLO

Fig. 24

Fig. 25

67

The Design Process

Expression The expressive approaches to word-sound compositions often employ different tools and techniques, which grants them a diverse and appealing look. Although they do not follow any strict rules, certain systems are observable. Without the clear intention, my solutions might look too random and it would be impossible to communicate anything to the reader. The notion that every spoken utterance is individualistic and each of us can articulate a word in a thousand ways, brought me to the idea of working with media that incorporate the aspect of unrepeatability. In the way that no single brushstroke can be reproduced in the same way, when looking at some of the outcomes the notion of uniqueness comes to mind. Instead of working with prosodic features only, some of these solutions employ more abstract concepts such as the general representation of sound and its ephemerality, trying to show speech as an expression of complex organisms.

During the first experiment a scanner was utilized to achieve organic distortion of letterforms (Fig. 26). Thanks to its space and time-based character, this tool perfectly aligns to the concept of the spoken word. The process of articulation is fixed to one moment and place, while the scanner similarly provides only limited time and space, depending on the resolution and size. Scanned experiments also employ the movement of the hand to demonstrate diverse emotional states connected to our bodily experience (shakiness, aggression, hesitation, etc.). In this manner, the indexical aspect of voice production is connected to the way our body acts during articulation. By altering speed and movement direction, it is possible to achieve various effects, such as character of intonation and pauses. Distorted letterforms can also indicate the length of pronunciation, repetition or jaw (vowel) openness.

Experiments realized with analogue media were also based on the idea of indexicality of letterforms. Using different kinds of brushes and other painting tools, the connection between hand movement and the production of letterforms was scrutinized (Fig. 27). Even though some of the findings might seem obvious, their presence in this research allows for more interesting conclusions.

For example, the question of how to show fast speech was ap-

68

The Design Process

proached by writing letters with a sponge brush (Fig. 28). It turned out that to achieve the notion of fast speech, aspects other than dynamic brushstrokes have to be present. In this case the proportion of letters along with the spacing and the connectedness of individual letters comes into play. It is important to point out that in order to read fast, letterforms should be as legible as possible, which would imply that the most common typeface in tight spacing could be instanly rendered as rapid articulation.

Fig. 26

Fig. 27

Fig. 28

69

The Design Process

In another experiment, a plastic bag was used as a medium that could be moulded in order to achieve the spatial depth and distortion of the letterforms. By such means intonation and phrasing can be depicted, along with the length and the type of articulation. When letters are incomplete, they also suggest imperfect pronunciation. The blur effect was employed as a tool to accentuate this issue even more. Blurriness demonstrates how particular letters, when articulated, blend into one sound. On the other hand, focused parts indicate lexical stress or the fact that the pronunciation of two individual letters is separated. Ultimately, all the letters do not have to be legible in order to decode the whole word (Fig. 29).

The next series was also focused on showing voice (and sound) as a complex and organic phenomenon. Stencils were used, in combination with spray paint, to create a wide range of expressive compositions. The main focus was on vowels being the main carrier of vocal alterations in an utterance. Newly developed glyphs were then used to substitute common letterforms in a word, indicating departure from ordinary speech. These kinds of sound notations are sometimes hard to decipher, but, on the other hand, provide more freedom for subjective renderings. Abstract qualities of sound such as temporality, flow, noise or irregularity are clearly observable on some examples (Fig. 30). In a similar manner, fabric was used to create another set of vowel alternates.

Ink was applied on the cut-outs of letters, which were then stamped on a paper. By moving them across the paper plane, their movement is captured with a lot of unintentional details that actually enhance the notion of an organic flow. While some of the solutions mark gentle and almost unnoticeable sounds, others show strong and decisive representations of sound waves (Fig. 31).

Fig. 29

70

The Design Process

Fig. 30

Fig. 31

Selected outcomes of expressive experiments were then combined to create a poster series for an artificial Sound Poetry Festival. This series should demonstrate the most successful experiments, applied to a field with a clear connection to speech and oral performance. The intention was that these posters should work as an advertisement for this kind of festival and act as a notation for an oral performance at the same time (Fig. 32).

71

The Design Process

Fig. 32

72

Conclusion

CONCLUSION

73

The Process Documentation

74

Conclusion

Many linguists and artists throughout history have addressed the relationship between spoken and written language. Linguists, who are especially interested in the ever-changing spoken form, constitute an opposition to typographers and researchers focused on the constant written form. Because our environment is becoming more and more visual, the written word has the main role in modern communication, despite our everyday exposure to both forms. The contrast between the two forms of communication aroused my interest in addressing this issue from the position of a graphic designer. The field of sound poetry emerged as an ideal choice for the application of both coded and expressive systems as a form of notation for oral performance.

The first part of the design process was devoted to coded systems, while the goal was to enrich the common Latin alphabet by adding more options of writing a particular letter. It emerged that distorting proportions of a letterform in combination with an adjusted compositional setting has a bigger impact on the perception of the auditory qualities than if other elements are attached to the letter. At the beginning, this kind of system seemed to be the most fitting solution to this problem, but as was shown later, it is almost impossible to address such a complex system entirely. Speech is an intricate expression of our bodies – following the Saussurean definition of parole – every single utterance is unique and individual. In addition, even just one person can say a word in a thousand different ways. Even the phonetic alphabet, which is the most complex code of language sounds, cannot be regarded as an absolute system covering all positions of human speech. Therefore it is an open question as to whether it is even possible to use closed coded systems for a description of the voice.

Without previous knowledge of the system created by Ernest Robson I came up with a similar solution, which gives me an interesting position for comparison. The basic idea that pitch is shown by elevating or depressing letters while intensity is indicated by a different boldness is basically the same. The most apparent distinction is that Robson’s solution is based on three available positions, sizes and thicknesses, but in my case the number of possible modifications is not exactly fixed. This fact contributes to the overall playfulness, but, on the other hand, depicts various positions of the voice more organically. Another dissimilarity is that Robson’s prosodynic print employs horizontal distortion in order to show the length of pronunciation, whereas in my

75

Conclusion

solution this aspect shows jaw openness. The length is simply shown by varying amounts of spacing among the letters. Finally, the outcome of my endeavour demonstrates a less structured approach, yet probably provides more options for an experimental oral performance.

The compositional experiments provided certain insights especially with regard to the position between words, and not the page itself. Mutual distance and position can cue changes in intonation and also dramatically alter the perception of a word and its meaning. It emerged that the spatial organization of words can also have an effect on their pronunciation, challenging the reader to reconsider the role of conventionalized ways of articulation.

As it turned out, expressive experiments were the more successful in terms of showing the vocal aspects. The issue of uniqueness was addressed by employing processes that cannot be reproduced in the very same way. The indexical aspect of the corporeal feeling is present when using brushes and fabric to create letterforms, while the connection to speech prosody as an index is developed. In this sense typography and voice production creates an analogy for both seeing and speaking. The connection between our body and the act of a performance is especially strong when we try to read some of the expressive solutions. Moving the head along the abstract shapes actually help to render the intonation patterns, offering the reader a fullbody experience.

In the case of some of the experiments with a scanner, a certain resemblance to spectrograms is evident. These spectrograms reveal a harmonic aspect of the voice, given by a computer algorithm and therefore constitute an accurate visual representation of the voice. Nevertheless, when these aesthetics are imported to expressive typography, the link between the written and the spoken word is established instantly. To summarize my findings, it must be mentioned that the resemblance to the truthful representation of sound (such as graphs, spectrograms etc.) helps to build the connection to the acoustic properties of a word. The moment of showing voice as an organic and ephemeral phenomenon also causes an increased awareness of various vocal renderings. Some of the outcomes may also suggest particular situations, such as the word “YES� when shouted on a busy railway station.

What I hoped to accomplish here is not a closed system that would exactly match ordinary spoken language. Most probably

76

Conclusion

an attempt such as this would not be successful, since the field of linguistics is vast and it would be almost impossible to cover all of its specifics and comprehensive demands in this time span.

Therefore the practical part of this master’s thesis project should be regarded as a visual exploration of what can be achieved by altering common letterforms to provide a specific kind of notation for an unusual oral performance, or to depict some of the auditory qualities that are connected to our perception of voice and sound in general. This project represents a journey without a certain end, aiming to visually capture some of the invisible concepts we know. Similarly as our language is evolving, new approaches could be developed offering a wide range of possibilities for a further investigation.

77

Endnotes

Endnotes 1 The fact that even neutral looking typefaces can be regarded as having a ideological subtext is discussed in a paper by Kinross, Robin. “The Rhetoric of Neutrality.”

Design Issues Vol. 2, No. 2, Autumn, 1985, p. 18–30.

2 Brownie, Barbara. “The Semiotics of Typography.”

Gestalt Perception of Fluid Typography. Unpublished, 2009, p. 3 3 Saussure, Ferdinand de. Course in General Linguistics.

New York: Philosophical Library, 1959, p. 66 4 Artists such as Jan Tschichold, Herbert Bayer, Velimir Khlebnikov and Ilja Zdanevich addressed the relations between form and content of typography in their essays 5 Hill, Charles A., and Marguerite H. Helmers. Defining Visual Rhetorics. Mawhaw, New Jersey: Lawrence Erlbaum, 2004, p. 15

12 Saussure, Ferdinand de. Course in General Linguistics. New York: Philosophical Library, 1959, p. 10 13 Ibid. p. 15 14 Ibid, p. 33 15 Ibid. p. 24 16 Saussure in this context provides division to ideographic and phonetic writing, while syllabic writing is not included in his discussion 17 Woods, Christopher, Geoff Emberling and Emily Teeter. Visible Language: Inventions of Writing in the Ancient Middle East and beyond. Chicago: Oriental Institute of the University of Chicago, 2010, p. 23 18 Heterographs are groups of words that have identical pronunciation but different spelling and meaning

6 Chandler, Daniel. Semiotic for Beginners. 2013. http://users.aber.ac.uk/dgc/Documents/S4B/sem02. html (visited on 20/5/2013)

19 An extensive overview of so called ‘new alphabets’ can be found in a book by Spencer, Herbert. The visible word. Problems of legibility. London 1968.

7 Hill, Charles A., and Marguerite H. Helmers. Defining Visual Rhetorics. Mawhaw, New Jersey: Lawrence Erlbaum, 2004, p. 16.

20 Fox, Anthony. Prosodic Features and Prosodic Structure: The phonology of Suprasegmentals.

Oxford, New York: Oxford University Press, 2000, p. 1

8 Barthes, Roland. Image, Music, Text. Ed. and trans.

Stephen Heath. London: Fontana Press, 1977, p. 42

21 Ibid. p. 9

9 Ibid, p. 7

22 Ramachandran, Vilayanur. From the transcription of BBC Reith Lectures 3/2003

10 Chandler, Daniel. Semiotic for Beginners. 2013. http://users.aber.ac.uk/dgc/Documents/S4B/sem08. html (visited on 20/5/2013)

23 Facial expressions composed of punctuation marks or other glyphs

11 Barthes, Roland. Image, Music, Text. Ed. and trans.

Stephen Heath. London: Fontana Press, 1977, p. 40

24 Brownie, Barbara. The Semiotics of Typography in Gestalt Perception of Fluid Typography. Unpublished. p. 9

78

Endnotes

25 Leeuwen, Theo van. “Towards a Semiotics of Typography.” Information Design Journal + Document Design 14.2, 2006, p. 142 26 Ehses, Hanno, Lupton, Ellen. Rhetorical Handbook: An Illustrated Manual for Graphic Designers. Halifax, Nova Scotia: Design Division, Nova Scotia College of Art and Design, 1988, p. 2 27 Ibid, p. 14

37 Lyrical ideograms also called by Apollinaire as ‘figurative’ or ‘ideogrammatic’ poems. Spencer, Herbert. Pioneers of Modern Typography. Cambridge, Massachusetts: MIT Press, 2004., p. 17 38 Bohn, Willard. The Aesthetics of Visual Poetry, 1914–1928. Cambridge: Cambridge University Press, 1986, p. 5

28 Ibid, p. 16

39 Saussure, Ferdinand de. Course in General Linguistics. New York: Philosophical Library, 1959, p. 34

29 Leeuwen, Theo van. “Towards a Semiotics of Typography.” Information Design Journal + Document Design 14.2, 2006, p. 139

40 Perloff, Marjorie and Craig Dworkin. The Sound of Poetry / The Poetry of Sound. Chicago: University of Chicago Press, 2009, p. 278

30 Lakoff, George, and Mark Johnson. Metaphors We Live By. Chicago: University of Chicago Press, 1980, p. 5

41 Bohn, Willard. The Aesthetics of Visual Poetry, 1914–1928. Cambridge: Cambridge University Press, 1986, p. 19

31 Johnson, Mark. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press, 1987, p. 136

42 Scholz, Christian. Relations between sound poetry and visual poetry: The path from the optophonetic poem to the multimedia text. 2001. http://www.dadacompanion.com/hausmann/archive/scholz_2001.pdf (visited on 20/5/2013)

32 Leeuwen, Theo van. “Towards a Semiotics of Typography.” Information Design Journal + Document Design 14.2, 2006, p. 146 33 Strizver, Ilene. Type Rules! Cincinnati, Ohio: North Light Books, 2001, p. 43

43 Drucker, Johanna. The Visible Word: Experimental Typography and Modern Art, 1909–1923. Chicago: University of Chicago Press, 1994, p. 118

34 The term typeface persona is commonly used in the field of typographic research to describe the personality, atmosphere or feeling evoked by a certain typeface

44 As cited in Drucker, Johanna. The Visible Word: Experimental Typography and Modern Art, 1909– 1923. Chicago: University of Chicago Press, 1994, p. 117

35 Mackiewicz, Jo. “How to Use Five Letterforms to Gauge a Typeface’s personality: A ResearchDriven Method.” Journal of Technical Writing and Communication 35.3, 2005, p. 291–315.

45 Wendt, Larry. “Sound Poetry: I. History of Electro– Acoustic Approaches II. Connections to Advanced Electronic Technologies.” Leonardo Vol. 18 No. 1, 1985, p. 11

36 Kinross, Robin. “The Rhetoric of Neutrality.” Design Issues Vol. 2, No. 2, Autumn, 1985, p. 18–30, p. 26

46 Perloff, Marjorie and Craig Dworkin. The Sound of Poetry / The Poetry of Sound. Chicago: University of Chicago Press, 2009, p. 239–241

79

Endnotes

47 Cheek, Cris. Bob Cobbing. http://www.southampton. ac.uk/~bepc/forum/cheek_cobbing.htm (visited on 20/5/2013) 48 Mayer, Peter. Alphabetical and Letter Poems: A Chrestomathy. London: The Menard Press, 1978, p. 99.

49 Wendt, Larry. “Sound Poetry: I. History of Electro– Acoustic Approaches II. Connections to Advanced Electronic Technologies.” Leonardo Vol. 18 No. 1, 1985, p. 13 50 McCaffery, Steve. Seven Pages Missing, vol. 1, Selected Texts 1969–1999. Toronto: Coach House Books, 2000, p. 446 51 Zaum is a term describing experiments connected to a invention of an imaginative language of Russian Futurist poets 52 In this case diagram can be understood as a symbolic representation of prosodic features 53 scribing tool kit mainly used by engineers 54 Wendt, Larry. “Sound Poetry: I. History of Electro– Acoustic Approaches II. Connections to Advanced Electronic Technologies.” Leonardo Vol. 18 No. 1, 1985, p. 12

80

The Process Documentation

BIBLIOGR APHY

81

The Process Documentation

82

Bibliography

Bohn, Willard The Aesthetics of Visual Poetry, 1914–1928. Cambridge: Cambridge University Press, 1986.

Brownie, Barbara

“The Semiotics of Typography.” Gestalt Perception of Fluid Typography. Unpublished, 2009.

Brumberger, Eva

“The Rhetoric of Typography: The Awareness and Impact of Typeface Appropriateness.” Technical Communication 50.2, 2003, p. 224–231.

“The Rhetoric of Typography: The Persona of Typeface and Text.” Technical Communication 50.2, 2003, p.

206–223.

Drucker, Johanna

The Visible Word: Experimental Typography and Modern Art, 1909–1923. Chicago: University of Chicago Press, 1994.

Ehses, Hanno and Ellen Lupton

Rhetorical Handbook: An Illustrated Manual for Graphic Designers. Halifax, Nova Scotia: Design Division, Nova Scotia College of Art and Design, 1988.

Fox, Anthony

Prosodic Features and Prosodic Structure: The phonology of Suprasegmentals. Oxford, New York: Oxford University Press, 2000.

Gerstner, Karl

Kompendium für Alphabeten. Sulgen, Zurich: Verlag Niggli AG, 2000.

Hill, Charles A. and Marguerite H. Helmers

Defining Visual Rhetorics. Mawhaw, New Jersey: Lawrence Erlbaum, 2004.

Johnson, Mark

The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press, 1987.

Kinross, Robin

“The Rhetoric of Neutrality.” Design Issues Vol. 2, No. 2, Autumn, 1985, p. 18–30.

83

Bibliography

Klanten, Robert

Type One, Discipline and Progress in Typography, Berlin: Die Gestalten Verlag, 2004.

Lakoff, George, and Mark Johnson

Metaphors We Live By. Chicago: University of Chicago Press, 1980.

Leeuwen, Theo van

“Towards a Semiotics of Typography.” Information Design Journal + Document Design 14.2, 2006, p.

139–55.

Mackiewicz, Jo

“How to Use Five Letterforms to Gauge a Typeface’s personality: A Research-Driven Method.” Journal of Technical Writing and Communication 35.3, 2005, p.

291–315.

Mayer, Peter

Alphabetical and Letter Poems: A Chrestomathy. London: The Menard Press, 1978.

McCaffery, Steve

Seven Pages Missing, vol. 1, Selected Texts 1969–1999.

Toronto: Coach House Books, 2000.

Perloff, Marjorie and Craig Dworkin

The Sound of Poetry / The Poetry of Sound. Chicago: University of Chicago Press, 2009.

Robson, Ernest

“Research of the Sounds of Literature: Formant Music and a Prosodic Notation for Performance.” Leonardo Vol. 20 No. 2, 1987, p. 131–138.

Saussure, Ferdinand de

Course in General Linguistics. New York: Philosophical Library, 1959.

Spencer, Herbert

Pioneers of Modern Typography. Cambridge, Massachusetts: MIT Press, 2004.

Strizver, Ilene

Type Rules!. Cincinnati, Ohio: North Light Books, 2001

84

Bibliography

Wendt, Larry

“Sound Poetry: I. History of Electro–Acoustic Approaches II. Connections to Advanced Electronic Technologies.” Leonardo Vol. 18 No. 1, 1985, p. 11–23.

Woods, Christopher, Geoff Emberling and Emily Teeter

Visible Language: Inventions of Writing in the Ancient Middle East and beyond. Chicago: Oriental Institute of the University of Chicago, 2010.

Internet Sources Bulatov, Dmitry

“GLUKHOMANIA.RU” An Electronic Museum of Lingua-Acoustic Space.

Source: http://glukhomania.ncca-kaliningrad.ru/ (visited on 20/5/2013)

Chandler, Daniel

Semiotic for Beginners. 2013

Source: http://users.aber.ac.uk/dgc/Documents/S4B/ (visited on 20/5/2013)

Cheek, Cris

Bob Cobbing.

Source: http://www.southampton.ac.uk/~bepc/forum/cheek_ cobbing.htm (visited on 20/5/2013)

Ramachandran, Vilayanur

Transcription of BBC Reith Lectures 3/2003.

Source: http://www.bbc.co.uk/radio4/reith2003/lecture1.shtml (visited on 20/5/2013)

Scholz, Christian

Relations between sound poetry and visual poetry: The path from the optophonetic poem to the multimedia text.

2001.

Source: http://www.dada-companion.com/hausmann/archive/ scholz_2001.pdf (visited on 20/5/2013)

85

Bibliography

Image Sources Fig. 1. Kurt Schwitters, Systemschrift, 1927

Source: http://media.smashingmagazine.com/wp-content/ uploads/2012/02/wwi-02_Schitters.jpg

Fig. 2. Robert Massin, From Eugene Ionesco‘s La Cantatrice Chauve (The Bald Soprano), 1964 Source: http://farm3.staticflickr.com/2245/2051611164_ a5f39da0cf_o.jpg

Fig. 3. Alexander Rodchenko, Poster for Soviet Union propaganda, 1924 Source: http://fiktura.files.wordpress.com/2010/04/alexanderrodchenko.jpg

Fig. 4. Stephane Mallarmé, Un Coup de Dés Jamais N’Abolira Le Hasard, 1897 Source: http://writing.upenn.edu/library/Mallarme-Stephen_ Coup_1914.pdf

Fig. 5. Filipo Tommaso Marinetti, Zang Tumb Tumb, 1914

Source: http://ecx.images-amazon.com/images/ I/71U4J24Ov%2BL._SL1071_.jpg

Fig. 6. Bob Cobbing, From Shakespear kaku, 1972

Source: Wendt, Larry. “Sound Poetry: I. History of Electro–Acoustic Approaches II. Connections to Advanced Electronic Technologies.” Leonardo Vol. 18 No. 1, 1985, p. 12

Fig. 7. Steve McCaffery, From CARNIVAL the second panel, 1970–75 Source: http://archives.chbooks.com/online_books/carnival/ images/2a_02.gif

Fig. 8. Joshua Steele, Transcription of Pope’s Happiness, 1775

Source: http://habla.dc.uba.ar/gravano/ith-2012/archivos-04/ JoshuaSteel.pdf

86

Bibliography

Fig. 9. Ernst and Marion Robson, Poem Crows in prosodynic script, 1946

Source: Robson, Ernest. “Research of the Sounds of Literature: Formant Music and a Prosodic Notation for Performance.”

Leonardo Vol. 20 No. 2, 1987, p. 132.

Fig. 10. Pierre di Sciullo, Le Quantage, 1989

Source: http://www.quiresiste.com/medias/QUAN1.GIF

Fig. 11. Meggi Zummstein, Visualisierung von Sprache (Visualization of Language), 2001 Source: Meggi Zummstein’s diploma thesis (pdf)

Fig. 12. Raoul Hausmann, kp’erioum, 1919

Source: http://upload.wikimedia.org/wikipedia/en/d/d4/Kp%27_ erioum-Hausmann.jpg

Fig. 13–32. Ondřej Jelínek, The Spoken Word in Typography, 2013

87

THE PROCESS DOCUMENTATION

Inspiration

91

The Process Documentation

Research

The Process Documentation

Das Lautstärken- und Stimmhaftigkeits-Alphabet

Der höchste Punkt entspricht der Lautstärke (dunkel = laut).

Die Breite der Buchstaben ergibt sich aus der hörbaren Präsenz des Buchstabens.

92

Research

The Process Documentation

FILTERKOMBINATION AUS DEM GESTALTUNGSINSTRUMENT

0.00 Spracheingabe und Zuweisung des Schriftschnittes

2.01 Lautstärke Schriftgrösse richtet sich nach der Lautstärke

2.03 Intensität Sprechintensität ergibt die Strichdicke

3.03 Artikulation Unartikulierte Buchstaben werden verstümmelt

3.04 Redefluss Gebunden sprechen verbindet die Buchstaben, abgehackt zerstückelt sie

4.03 Höhenkurve Die Tonhöhe generiert ihr Sprachgebirge

3.01 Stimmhaftigkeit = Proportion

BETONUNG

3.02 Schwingung = Proportion

2.01 Lautstärke = Schriftgrad

3.03 Artikulation (überschlagend) = Gestalt

2.02 Lautstärkengebilde = Grösse

3.03 Artikulation (flüsternd) = Gestalt

2.03 Intensität = Stärke

3.04 Redefluss (gebunden) = Gestalt

2.04 Unterstreichen = Stärke

3.04 Redefluss (abgehackt) = Gestalt

2.05 Eckpunkte = Betonung

KLANGFARBE

93

Research

94

The Process Documentation

Research

95

The Process Documentation

Research

96

The Process Documentation

Research

97

The Process Documentation

Research

98

The Process Documentation

Research

99

The Process Documentation

Research

100

The Process Documentation

Research

101

The Process Documentation

Research

102

The Process Documentation

Research

103

The Process Documentation

Research

104

The Process Documentation

Research

105

The Process Documentation

Research

106

The Process Documentation

Research

107

The Process Documentation

Research

108

The Process Documentation

Research

109

The Process Documentation

Research

110

The Process Documentation

Research

111

The Process Documentation

Notes

112

The Process Documentation

Notes

113

The Process Documentation

Notes

114

The Process Documentation

Notes

115

The Process Documentation

Notes

116

The Process Documentation

Notes

117

The Process Documentation

Sketches

118

The Process Documentation

Sketches

119

The Process Documentation

Sketches

120

The Process Documentation

Sketches

121

The Process Documentation

Experiments

122

The Process Documentation

Experiments

123

The Process Documentation

Experiments

124

The Process Documentation

Experiments

125

The Process Documentation

Experiments

126

The Process Documentation

Experiments

127

The Process Documentation

Experiments

128

The Process Documentation

Experiments

129

The Process Documentation

Experiments

130

The Process Documentation

Experiments

131

The Process Documentation

Experiments

132

The Process Documentation

Experiments

133

The Process Documentation

Experiments

134

The Process Documentation

Experiments

135

The Process Documentation

Experiments

136

The Process Documentation

Experiments

137

The Process Documentation

Experiments

138

The Process Documentation

Experiments

139

The Process Documentation

Experiments

140

The Process Documentation

Experiments

141

The Process Documentation

Experiments

142

The Process Documentation

Experiments

143

The Process Documentation

Experiments

144

The Process Documentation

Experiments

145

The Process Documentation

Experiments

146

The Process Documentation

Experiments

147

The Process Documentation

Experiments

148

The Process Documentation

Experiments

149

The Process Documentation

Experiments

The Process Documentation

A

H

T

W D

O

E

S

M

A

R

S

E

L

L U

S

A

H

T

W D

O

E

S

M

A

R

S

E

L

L U

S

A

H

T

W

D

O

E

S

M

A

H

A

R

S

E

L

L

U

L

U

S

T

W

D

150

O

E

S

M

A

R

S

E

L

S

Experiments

A W

A W

L L A

L L A

A L W

A L W

The Process Documentation

C E

L OO K

E? LI K

C E

L OO K

E? LI K

L A C

E

L

O

L A C

E

L

O

151

O

K

O

K

L

L

I K

E

?

I K

E

?

Experiments

152

The Process Documentation

Experiments

153

The Process Documentation

Experiments

154

The Process Documentation

Experiments

155

The Process Documentation

Experiments

156

The Process Documentation

Experiments

157

The Process Documentation

Experiments

The Process Documentation

H E Y

HOW

158

Y OU

K I

B

D

O

S

Y

S

DO

I NG?

Experiments

The Process Documentation

H

E

E

E Y

K

E

E P

C

159

H I

L

L

IN

Experiments

nnnnnnnnnnnnnnnnnnnnnnnnnnnnnno nnnnnnnnnnnnnnnnnnnnnnnnnnnnnoo nnnnnnnnnnnnnnnnnnnnnnnnnnnnooo nnnnnnnnnnnnnnnnnnnnnnnnnnnoooo nnnnnnnnnnnnnnnnnnnnnnnnnnooooo nnnnnnnnnnnnnnnnnnnnnnnnnoooooo nnnnnnnnnnnnnnnnnnnnnnnnooooooo nnnnnnnnnnnnnnnnnnnnnnnoooooooo nnnnnnnnnnnnnnnnnnnnnnooooooooo nnnnnnnnnnnnnnnnnnnnnoooooooooo nnnnnnnnnnnnnnnnnnnnooooooooooo nnnnnnnnnnnnnnnnnnnoooooooooooo nnnnnnnnnnnnnnnnnnooooooooooooo nnnnnnnnnnnnnnnnnoooooooooooooo nnnnnnnnnnnnnnnnooooooooooooooo nnnnnnnnnnnnnnnoooooooooooooooo nnnnnnnnnnnnnnooooooooooooooooo nnnnnnnnnnnnnoooooooooooooooooo nnnnnnnnnnnnooooooooooooooooooo nnnnnnnnnnnoooooooooooooooooooo nnnnnnnnnnooooooooooooooooooooo nnnnnnnnnoooooooooooooooooooooo nnnnnnnnooooooooooooooooooooooo nnnnnnnoooooooooooooooooooooooo nnnnnnooooooooooooooooooooooooo nnnnnoooooooooooooooooooooooooo nnnnooooooooooooooooooooooooooo nnnoooooooooooooooooooooooooooo nnooooooooooooooooooooooooooooo noooooooooooooooooooooooooooooo

160

The Process Documentation

yeeeeeees yyeeeeees yyyeeeees yyyyeeees yyyyyeees yyyyyyees yyyyyyyes yyyyyyess yyyyyesss yyyyessss yyyesssss yyessssss yesssssss yeessssss yeeesssss yeeeessss yeeeeesss yeeeeeess yeeeeeees yyeeeeess yyyeeesss yyyyessss

Experiments

NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNO NNNNNNNNNNNNNNNNNNNNNNNNNNNNNOO NNNNNNNNNNNNNNNNNNNNNNNNNNNNOOO NNNNNNNNNNNNNNNNNNNNNNNNNNNOOOO NNNNNNNNNNNNNNNNNNNNNNNNNNOOOOO NNNNNNNNNNNNNNNNNNNNNNNNNOOOOOO NNNNNNNNNNNNNNNNNNNNNNNNOOOOOOO NNNNNNNNNNNNNNNNNNNNNNNOOOOOOOO NNNNNNNNNNNNNNNNNNNNNNOOOOOOOOO NNNNNNNNNNNNNNNNNNNNNOOOOOOOOOO NNNNNNNNNNNNNNNNNNNNOOOOOOOOOOO NNNNNNNNNNNNNNNNNNNOOOOOOOOOOOO NNNNNNNNNNNNNNNNNNOOOOOOOOOOOOO NNNNNNNNNNNNNNNNNOOOOOOOOOOOOOO NNNNNNNNNNNNNNNNOOOOOOOOOOOOOOO NNNNNNNNNNNNNNNOOOOOOOOOOOOOOOO NNNNNNNNNNNNNNOOOOOOOOOOOOOOOOO NNNNNNNNNNNNNOOOOOOOOOOOOOOOOOO NNNNNNNNNNNNOOOOOOOOOOOOOOOOOOO NNNNNNNNNNNOOOOOOOOOOOOOOOOOOOO NNNNNNNNNNOOOOOOOOOOOOOOOOOOOOO NNNNNNNNNOOOOOOOOOOOOOOOOOOOOOO NNNNNNNNOOOOOOOOOOOOOOOOOOOOOOO NNNNNNNOOOOOOOOOOOOOOOOOOOOOOOO NNNNNNOOOOOOOOOOOOOOOOOOOOOOOOO NNNNNOOOOOOOOOOOOOOOOOOOOOOOOOO NNNNOOOOOOOOOOOOOOOOOOOOOOOOOOO NNNOOOOOOOOOOOOOOOOOOOOOOOOOOOO NNOOOOOOOOOOOOOOOOOOOOOOOOOOOOO NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

161

The Process Documentation

YEEEEEEES YYEEEEEES YYYEEEEES YYYYEEEES YYYYYEEES YYYYYYEES YYYYYYYES YYYYYYESS YYYYYESSS YYYYESSSS YYYESSSSS YYESSSSSS YESSSSSSS YEESSSSSS YEEESSSSS YEEEESSSS YEEEEESSS YEEEEEESS YEEEEEEES YYEEEEESS YYYEEESSS YYYYESSSS

Experiments

162

The Process Documentation

Experiments

163

The Process Documentation

Experiments

164

The Process Documentation

Experiments

165

The Process Documentation

Experiments

166

The Process Documentation

Experiments

167

The Process Documentation

Experiments

168

The Process Documentation

Experiments

169

The Process Documentation

Experiments

170

The Process Documentation

Experiments

171

The Process Documentation

Experiments

172

The Process Documentation

Experiments

173

The Process Documentation

Experiments

174

The Process Documentation

Experiments

175

The Process Documentation

Experiments

176

The Process Documentation

Experiments

177

The Process Documentation

Experiments

178

The Process Documentation

Experiments

179

The Process Documentation

Experiments

180

The Process Documentation

Experiments

181

The Process Documentation

Experiments

182

The Process Documentation

Experiments

183

The Process Documentation

Experiments

184

The Process Documentation

Experiments

185

The Process Documentation

Experiments

186

The Process Documentation

Experiments

187

The Process Documentation

Experiments

188

The Process Documentation

Experiments

189

The Process Documentation

Experiments

190

The Process Documentation

Experiments

191

The Process Documentation

Experiments

192

The Process Documentation

Experiments

193

The Process Documentation

Experiments

194

The Process Documentation

Experiments

195

The Process Documentation

Experiments

196

The Process Documentation

Experiments

197

The Process Documentation

Experiments

198

The Process Documentation

Experiments

199

The Process Documentation

Experiments

200

The Process Documentation

Experiments

201

The Process Documentation

Experiments

202

The Process Documentation

Experiments

203

The Process Documentation

Experiments

204

The Process Documentation

Experiments

205

The Process Documentation

Experiments

206

The Process Documentation

Experiments

207

The Process Documentation

Experiments

208

The Process Documentation

Experiments

209

The Process Documentation

Experiments

210

The Process Documentation

Experiments

211

The Process Documentation

Experiments

212

The Process Documentation

Experiments

213

The Process Documentation

Experiments

214

The Process Documentation

Experiments

215

The Process Documentation

Experiments

216

The Process Documentation

Experiments

217

The Process Documentation

Experiments

218

The Process Documentation

Experiments

219

The Process Documentation

Experiments

S F O E U P S N O T D E I T V R A Y L

220

The Process Documentation

Experiments

221

The Process Documentation

Experiments

222

The Process Documentation

Experiments

223

The Process Documentation

Experiments

224

The Process Documentation

Experiments

225

The Process Documentation

Experiments

226

The Process Documentation

The Spoken Word in Typography

Master’s Thesis by Ondrej Jelinek

Mentors

Prof. Michael Renner Paloma López Grüninger, PhD Jiri Oplatek

Concept and Design

Ondrej Jelinek

Typeface

Euclid BP Bold, Univers LT Condensed, Adobe Caslon Pro Master of Arts in Visual Communication and Iconic Research University of Applied Sciences and Arts Northwestern Switzerland Academy of Arts and Design HGK FHNW Visual Communication Institute The Basel School of Design Vogelsangstrasse 15 CH-4058 Basel © September 2013

Master of Arts in Visual Communication and Iconic Research University of Applied Sciences and Arts Northwestern Switzerland Academy of Arts and Design HGK FHNW Visual Communication Institute The Basel School of Design Š 2013