BRESSEM-PANEL

GESPIN – GESTURE & SPEECH IN INTERACTION – Poznań, 24-26 September 2009

Towards a grammar of gesture Panel organizers: Jana Bressem, Silva H. Ladewig, Cornelia Müller, Susanne Tag European University Viadrina, Frankfurt/Oder

Participants: Jana Bressem*, Ellen Fricke*, Silva H. Ladewig*, Irene Mittelberg**, Cornelia Müller*, Susanne Tag* *European University Viadrina, Frankfurt/Oder **Human Technology Centre, RWTH Aachen Abstract This session presents a linguistic and semiotic perspective on gestures as pursued in the VW research project „Towards a grammar of gesture: Evolution, brain and linguistic structures“ which aims at an encompassing account of the structural properties of gestures. The core of the project is thereby on a descriptive account of the fundamental semantic and formal structures in human gestures (Kendon 2004, Müller 1998, 2004, Calbris 1990) and their syntactic integration with speech: in short, the foundations of a grammar of gesture. Showing that gestures are highly structured signs, which are syntactically integrated into language – for instance, as attributive constructions – challenges the traditional concept of language as a closed system of vocal signs and imply that the very nature of language is multimodal.

Such a perspective on gestures not only serves as a point of departure for further studies into gesture analysis in fields such as neurology and evolutionary anthropology, but will also inform fields that have discovered gestures as a revelatory ‘window onto thought’ (McNeill 1992, 2005), such as more general cognitive psychology, artificial intelligence, and communication studies, but which all currently suffer from a lack of shared scientifically grounded knowledge of gestures.

With this panel we would like to present an overview of the linguistic and semiotic sub- projects addressing the following topics:

1

Semantics: Specifying principles of meaning creation in gestures

Gestures can, amongst other things, be used to refer to an unlimited number of concrete as well as abstract entities or events. However, in the meaning-making processes, gestures are based on a limited set of mimetic devices, termed “gestural modes of representation”, which imply different cognitive processes of sign creation (Müller 1998, Müller et al. in prep.). How the hands turn into gestures or signs by making use of these different modes will be presented in a first talk. In a second talk we would like to explore the question of whether gestures of the concrete and of the abstract (Müller 1998) differ according to their form. First results of the experiment conducted in order to approach this question will be presented.

GESPIN – GESTURE & SPEECH IN INTERACTION – Poznań, 24-26 September 2009

2

Aspects of Form: forms, meaning, and the integration of gestures with language

When watching people gesture across different discursive contexts, it turns out that a limited set of gestures recurs showing a stable form-meaning relationship (Kendon 2004, Ladewig in prep., Müller 2004, Teßendorf in prep.). We will give first results of our documentation of recurrent gestures, their forms, meanings and functions German speakers make use of (Müller, Bressem, Fricke in prep).

How co-speech gestures may be combined to more complex patterns that build up larger gestural units shall be presented in a second talk. The first results of a documentation of basic forms of gesture combinations are based on analyses of structural relations between two or more gestures comprising temporal relations (Fricke 2008; Tag in prep.) as well as formal relations between the combined gestures (Müller & Tag in prep.).

3

Integration of gestures with speech

Theoretical as well as empirical foundations for a multimodal grammar of German will be given (Fricke 2007, 2008, in prep.). The talk aims at an inclusion of gestures into syntactic, semantic, and pragmatic descriptions of utterance sequences by showing, for instance, that gestures take over attributive functions. These findings suggest that speakers make use of from the different modalities they have at hand according to their communicative needs.

The panel will be opened by a talk addressing the overall structure of the project along with its involved disciplines (linguistics, semiotics, neurology, and evolutionary anthropology) and its three key topics: the multimodality of language, neurocognitive foundations of gestures as part of human language, and the evolution of language.

Bibliography Bressem, J. (in prep.). Recurrent form features in coverbal gestures. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.

Calbris, G. (1990). The semiotics of French gestures. Bloomington: Indiana University Press.

Fricke, E. (2007). Origo, Gesten und Raum: Lokaldeixis im Deutschen. Berlin: Walter de Gruyter.

Fricke, E. (2008). Grundlagen einer multimodalen Grammatik des Deutschen: Syntaktische Strukturen und Funktionen.

Habilitation, Europa-Universität Viadrina, unpublished manuscript (will be published in 2009 by de Gruyter).

Fricke, E. (in prep.). Kinaesthemes: Type construction and semantic loading in co-speech gestures. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.

Kendon, A. (2004). Gesture. Visible action as utterance. Cambridge: Cambridge University Press.

Ladewig, S. H. (in prep.). The family of the cyclic gesture – systematic variation of form and context. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.

Ladewig, S. H. & J. Bressem (submitted) Discovering structures in gestures on the basis of the four parameters of Sign Language.

McNeill, D. (1992). Hand and Mind. What gestures reveal about thought. Chicago: University of Chicago Press.

Mittelberg, I. (2008). Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar. In A. Cienki and C. Müller (eds.), Metaphor and Gesture. Amsterdam: Benjamins, 115-154.

Mittelberg, I. (in press). Geometric and image-schematic patterns in gesture space. In V. Evans and P. Chilton (eds.), Language, Cognition and Space: The State of the Art and New Directions. London: Equinox.

Müller, C. (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. (Vol. 1 of the series "Körper – Kultur – Kommunikation", edited by H. Kalverkämper, R. Krüger & R. Posner), Berlin Verlag: Berlin.

Müller, C. (2004). The Palm-Up-Open-Hand. A case of a gesture family? In C. Müller and R. Posner (eds.) The semantics and pragmatics of everyday gestures. The Berlin conference, Berlin: Weidler Buchverlag, 233–256.

Müller, C. (in prep.). Recurrence in gestures. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.

Müller, C., E. Fricke, H. Lausberg & K. Liebal (2005). Grant proposal "Towards a grammar of gesture: evolution, brain, and linguistic structures", Volkswagenstiftung.

Müller, C. , Bressem, J., Fricke, E. (in prep.). “No, not, none of that” - cases of exclusion and negation in gesture.

Müller, C., Fricke E., Ladewig, S. H., Mittelberg, I. & Teßendorf, S. (in prep.). Gestural Modes of Representation revisited.

Müller, C. & Tag, S. (in prep.). Combining gestures: complex patterns of coverbal gestures.

Tag, S. (in prep.). Simultaneous constructions in co-speech gestures. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.

Teßendorf, S. (in prep.). Pragmatic and metaphoric gestures – combining functional with cognitive approaches. In J. Bressem and S. H. Ladewig (eds.) Handmade patterns. Recurrent forms and functions of gestures.