reading - THE MULTIMODAL EXPERIENCE OF ART

Theoretical Perspective

Multimodality is diverse throughout the senses. There are many crossovers to be made so grounding myself in a selected number of sensory combinations is essential for the sake of keeping a cohesive argument.

p.356

Commentary on the intensity of music being met with more vigorous visual cues as the film and music progresses would mirror this observation in the context of music video/visuals. 

p. 357

Having different temporality between sound and image running on the same timeline could distort the modal crossovers...

GOOD WORD - Diegetic - In a film, diegetic sound would be heard by the characters and within the story, rather than targeted toward the audience.

Having a commentary on ways of perceiving an artistic piece multi-modally is valuable for the dissertation and film production. I am continuing my consideration of applying different cognitive methods to gauge reactions from my audience. By keeping records of the process of layering the sound and image and by applying some principles within this paper, I am narrowing down what needs to be achieved in my investigation. This will also aid my audiovisual storytelling methods.

For example, in Nanay's case study of choreography, applying motion at a different speed to the music playing can influence our multimodal perception. Alike to Chion's studies it brings into consideration the temporality of the senses and how this can be manipulated to different effects. Having sound and image at odds with each other can achieve certain effects such as surrealism and confusion. I can apply this to the scene with a strobe light. I believe it would be effective to juxtapose the fast moving light with slowed motion, it will be useful to my practical thought process for research and also the aesthetic for the final film. 

Considering the case studies within this paper works in my favour as well. When previously looking at visual stimuli for music, much of my studies had been on A/V, or application of visuals with digital precision and aesthetic. by comparing my work to dance choreography allows me to focus my studies on character movement and its relationship with the music, rather than abstract, code like apparitions.

How can gestural animations improve cognitive interaction to musical visuals?

How iterative processes in music video production can improve visuals and multimodal engagement with music

Nanay, B. (2012) ‘The Multimodal Experience of Art’, British Journal of Aesthetics, 52(4), pp. 353–363. Available at: https://discovery.ebsco.com/linkprocessor/plink?id=2bb61908-ad78-3b79-bf9f-3e4c0fcce695 (Accessed: 20 January 2022).

.