CONTEXTUAL REVIEW

My exploration will delve into cognitive responses to sound and music with a focus on gestural movements and animations. For a controlled contextual review, I will first examine instances of images triggering experience, followed by sound. By looking at these in their own respective modalities, connections will be drawn between the two. 

A principle within the MIT press of Experience: Culture, Cognition and the Common Sense frequently refers to projection; a causal event wherein a viewer/listener/experiencer can foresee events which happen next. The way individuals process information is all unique, but somewhat governed by the fact that ‘our experience of coherence and meaning is produced by activities of modelling - processes of sifting impressions, cascades of calculations, and attributions of significance carved from determinations of causality, which are in turn embedded in and generated by environments we have come to know’.

In a time where artistic multimodality may have more frequently manifested in still images, some interesting thoughts and responses were emerging. In the early 1900’s, Italian Futurist painter Gino Severini was depicting experience based on his own perception of particular events. Plastic analogies (a term Severini coined) were his methodology in mapping out ‘imagistic composites that combined two or more symbolic forms into a single image’(p57). The fact that movement, motion and even sound were being depicted in still images from 1914 is astounding and represents the importance people place on exploring cognition and how artistic practices are an outlet to record, analyse and discover.

In 1943, responses of using artistry as a way to gauge cognition and experience were emerging in the form of simple animations. Fritz Heider and Marianne Simmel experimented with endowing simple shapes with motion: “The projected image shows an outline of a rectangle with an opening on its right side; near this opening the shape of a triangle arrives. Soon a smaller triangle and a circular disk also enter the frame from above. The rectangle remains static, but the disk and triangles become quite literally animated, and each moves in a manner that comes to seem characteristic. That is, they are given movements that model different characters.’(p13) The aim was to research how viewers may perceive these shapes with attributes which go beyond their two dimensional limitations. A spectrum of titles to the research emerged, from ‘Perceptual of Geometrical Figures to Warring Triangles’.(p14) These titles alone aptly represent what the researchers were aiming to achieve. Representing a film ‘devoid of content’ yielding a range of fully fledged stories. The essay I refer to comes from the MIT press book, Experience, and the Heider-Simmel film is noted to have stock in contemporary studies of neuroscience, psychology, and the ‘aesthetics of ethics’(p14). The knowledge that artistic practices can be as informative to the studies of cognition and experience as scientific methods can, alongside the epistemology that 'Cognition implies a degree of plasticity for human psychophysical processes; in this sense, it can be as active and adaptable as consciousness itself'(p8) provides a good framework for my investigation as I elaborate into my own fusion of character animation and abstract representations of sound to stimulate genuine responses from viewers from a variety of sensory modalities. Furthermore, this research continues to be relevant to modern cognitive studies at MIT, with Josh Tenenbaum using the film within studies of Visual and Environmental Studies in 2014. ‘Modelling is a practice common to both artists and scientists, and as a customary activity of humans in general; our modelling behaviours distinguish us as a species. We model ideas, we model clay, we model with pictures, texts, clothing. What were Fritz Heider and Marianne Simmel modelling when they made their enduring film? The short answer is projection. But on a larger scale, they were modelling the complexity in the human psyche’. Here we see this film within a modern context; exploring the malleable nature of cognition as an artistic tool, these films and studies become a heuristic approach to cognitive science.

There are even links between the experience of film and the principles of animation. The 12 principles of animation have much correlation with how one may evoke genuine experience in a viewer.

Look at multimodal approaches THROUGH THE LENS of Italian Futurism and Plastic Analogies