Essay headers

 Psychology of sound and image

Inhibited senses (synesthesia & blindness)

Context

“Movies from America in particular were believed to propagate the concealed assumptions of Capitalism”

“Any view that is non reflective or non ironic is characterised as naive” (p.5)

“An apparent advantage to writing film from this perspective is that film writers may feel free to appropriate the language of psychoanalysis, Marxism, or any pop culture movement or special interest cause, without assuming responsibility for their theoretical imperatives. Such writers may claim adherence to no theory at all.” 

“Film is seen merely as a vehicle for revealing problems of social conflict and authority.” (p.7)

Film was propaganda

“By the mid eighties, a few courageous film theorists suggested that cognitive science might be a more productive path than the then pervasive, psychoanalytic/Marxist approach to film study.” (p.8)

“The makers of the movie have often spent many months and millions of dollars to achieve perfection of individual elements and overall form” (p.10)

“Hugo Muntersberg in 1915 predicted that film would become the domain of the psychologist. It has instead become the province for the entrepreneur.” (p.11)

“The motion picture can be thought of as a program. And it is more precisely a program than either a language or a mere set of stimuli. It is a very complex set of instructions utilising images, actions and sounds, a string of commands to attend to this now, in this light, from this angle, at this distance, and so forth”

“To recall earlier sequences and anticipate future ones” (p.12)

“The viewer can be thought of as a standard biological audio/video processor”

Diff between computer and brain

“Carrying out tasks like keeping track of expenses or balancing checkbooks”

“Good at guiding our movements through three dimensional space” (p.13)

The Reality of Illusion - An ecological approach to cognitive film theory

“Application of cognitive science to the field of films theory” (preface, x)

Multi modal

the human brain is primarily an image processor.

visuals are more concrete to the brain than words, they’re easier to remember, as well.

The visceral, emotional reactions that strong visuals can evoke are even quicker for our brains to process than emotion-neutral visuals — 13 milliseconds on average. Responding to sound is only slightly slower, at 146 milliseconds on average

the tasks involved in processing and enjoying music are distributed across several brain areas.”

A 2001 study at McGill College, cited by Mauk, found that subjects listening to music showed neural activity in the same areas of the brain that respond to othereuphoric stimuli like food, sex or drugs.In other words, our brains find music uniquely rewarding.

psychology professor Shinsuke Shimojo, whose research team explored ways in which stimulating one sense, such as hearing, could cause the brain to produce feedback that appealed to a related sense, like vision.

sound and vision don’t have a 1:1 correlation; instead, they interact with each other in highly complex ways. One of them is the ability for one input to provoke the other

https://blog.audiosocket.com/psychology-of-sound/

Film and music

Andre Bazin - “For Bazin, the film was neither a product of the mind nor a clash of concepts, but rather a photochemical record of reality.” (p.5) - film was not considered an art form.

The Reality of Illusion - An ecological approach to cognitive film theory

“Application of cognitive science to the field of films theory” (preface, x)

electronic music performances are often sensor or laptop-based, which are not always visible to the public and whose usage does not require big gestures and actions from the performer. 

an unclear cognitive link between the sound and the performer’s actions

https://research.gold.ac.uk/20878/6/AM_2017_ACM_compliant_cameraready_paper_36%20%281%29.pdf

The visuals are what the viewer tends to mostly focus on and the sound subconsciously alters how the visuals are perceived.”

since early 2007, the Internet has seen a 9900 percent increase in the use of visualized information

Ebendorf found a clear correlation between the way each video had been rated and the way the music accompanying it was rated when the two were experienced together.

Not only does sound affect what we see, but visuals affect what we hear, too.

https://blog.audiosocket.com/psychology-of-sound/

Any musician that reaches audiences of a certain size will eventually face the question of A/V accompaniment, regardless of whether visual presentation has been central to their work or not.

 I love to have the visuals connected to the music, synchronized and tight. In the best case, it should represent the sound on the visual level. 

Noemi Schipfer and Takami Nakamoto

Spectacle

While it isn’t subtle, spectacle is effective, stunning an audience’s senses and leaving a big impression. Often, video isn’t even necessary, as lights, especially strobes, can accomplish this on their own. And as lights get more technologically advanced, it becomes easier to do interesting things with them.

https://www.ableton.com/en/blog/performing-music-with-visuals/

Practical

having access to tutorials on digital musical instruments before the performance increased the audience’s know-how on the topic 

How clear is the relationship between the performer’s actions and, in our particular case, the audiovisual result?”.

https://research.gold.ac.uk/20878/6/AM_2017_ACM_compliant_cameraready_paper_36%20%281%29.pdf

one study asked students to remember many groups of three words each, such as dog, bike, and street. Students who tried to remember the words by repeating them over and over again did poorly on recall. In comparison, students who made the effort to make visual associations with the three words, such as imagining a dog riding a bike down the street, had significantly better recall.

https://www.psychologytoday.com/gb/blog/get-psyched/201207/learning-through-visuals

visual information—including the performer's sex, attractiveness, movement, and so on— could confound listeners' ability to make judgments of the quality of the music being performed. 

https://www.psychologytoday.com/gb/blog/get-psyched/201207/learning-through-visuals

AUDIO-VISION  SOUND ON SCREEN - MICHEL CHION

“We persist in ignoring how the soundtrack has modified perception”

“Each perception remains nicely in it’s own compartment” - as an audience, we never consider the interplay between the senses as sound and image are meticulously combined seamlessly.

“Identifying so called redundancy between the two domains and debating interrelations between forces”...”Which is more important, sound or image?”

(preface, xxvi)

Commentary on Bergman’s Persona - cutting sound from one scene sees a succession of 3 shots - “the entire sequence has lost its rhythm and unity”

“Sounds we didn’t especially hear when there was only sound emerge from the image like dialogue balloons in comics.”

(page 4)

“Audiovisual Illusion”

“Informative value with which a sound enriches a given image so as to create a definite impression”

Added value - Bergman “(eminently incorrect) impression that sound is unnecessary, that sound merely duplicates a meaning which in reality it brings about”

Image synchronism - (sounds happening in time on screen, blows, explosions etc.)

(page 5)

“Voice that is isolated in the sound mix like a solo instrument”...”sounds (music and noise) are merely an accompaniment.)

“You will first seek the meaning of the words, moving on to interpret the other sounds only when your interest in meaning has been satisfied”

(page 7)

“Music can directly express its participation in the feeling of a scene, by taking on the scene’s rhythm, tone, and phrasing”

“On the other hand, music can also exhibit conspicuous indifference to the situation, by progressing in a steady, undaunted, and ineluctable manner”

(page 8)

Anempathetic effect

“For example, in a very violent scene after the death of a character some sonic process continues, like the noise of a machine, the hum of a fan, a shower running, as if nothing had happened”

(page 9)

“Sound superimposed onto image is capable of directing our attention to a particular visual trajectory.”

(page 11

“Deaf people raised on sign language apparently develop a special ability to read and structure rapid visual phenomena”...”raises the question whether the deaf mobilize the same regions at the centre of the brain as hearing people do for sound - one of the many phenomena that lead us to question received wisdom about distinctions between the categories of sound and image”

(page 12)

(page 13) - THREE ASPECTS OF TEMPORALISATION - IMPORTANT

Tarkovski - called cinema “the art of sculpting in time”

The image projects onto them meaning they do not have at all by themselves

Sound more than image has the ability to saturate and short-circuit our perception (page 33)