Verbal and "nonverbal" expression in language production
David Vinson, PhD
Institute for Multimodal Communication, UCL Dept of Experimental Psychology
How do the different channels available to communicators trade off in expressing a desired message? When one channel is limited or altered, can we still convey messages as effectively by using other modes? And how do we adjust our communication to adapt?
In a seminal study by Rauscher et al (1996), speakers were restricted from making gestures, with consequences (difficulty of lexical access) that appeared to be limited to spatial content. However, most studies fail to consider the range of channels available to a speaker, even though we now know that communicators synchronise on a wide variety of dimensions ranging from social to lexical (e.g. Louwerse et al., 2012). And if people are naturally adaptive to limitations affecting one aspect of communication, we might expect them to accommodate via nonverbal expression such as gesture or facial expression. Pilot data from my lab suggest, however that this is not the case: restricting speech (by asking participants to whisper) led to reduced gesturing and greater dysfluency, as well as a change in the relationship between facial expressions and variation in acoustic properties. It remains unclear, however, what underlies the lack of adaptation or accommodation: projects in this area are intended to explore some of the different possibilities.
Individual projects can be flexible, depending on students' particular interest focused on specific aspects of verbal or nonverbal expression such as prosody, gesture, facial expression, movement, linguistic content and fluency, investigating how they may vary in restricted speech vs. an unrestricted control condition. Each project will require collecting data from a relatively small number of participants but a more substantial amount of detailed coding of video data (training will be provided).
Relevant articles:
Louwerse, M. M., Dale, R., Bard, E. G., & Jeuniaux, P. (2012). Behavior matching in multimodal communication is synchronized. Cognitive Science, 36(8), 1404-1426.
Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7(4), 226-231.