Final Dissertation corrections

For the final essay tutorials, I had queries about how I should conduct my primary research. I also needed to heavily refine what I had already written.

A survey did not seem a good way to gather results about my projection mapping experiments because there is no accurate way to gauge the videos in correlation with my studies of Chion. Therefore, I took to analysing my own work. The video was split into 3 examples representing different audiovisual fusion techniques found in film. By analysing these I would find my results.

With my already produced work it became clear that for other readers some of the points were convoluted and didn't link back to the question. Therefore it was important to comb through the writing and link back to the title. Many sentences required shortening and simplifying.

Music Video research

 Excerpt from a book about fungus and the complex networks it creates, able to supply nutrients to different parts of a forest depending on the ecosystem.

David Attenborough excerpt showing cordyceps, a natural fungal occurrence. the phenomenon is bizarre but has an aesthetic which would animate well and fit to the themes in the video.

Music video starter

The EP cover art gives an idea of the colours and aesthetic I will use moving forward. This project will most likely be mixed media using an amalgamation of rotoscoping, collage, 2D traditional animation and live photography.

The video will also include dynamic colour as an embodiment of the nature of the song.

 

CT Scan reference video

Animation to Psytrance music

Animated in After Effects, the background shifts from different nature photos changing in Opacity.

The face was hand drawn and coloured, scanned and put into After Effects.

Lighting was added to blend layers together.

WWW

Colour palette is good, bright and invokes themes of fungus/toxicity in nature

Tests coral and sky backgrounds, both of which blend well with the hand drawn illustrations

Worked well with real CT scan as a reference video

EBI

Extremely time consuming doing on Lightbox

Music Video Photography and Hand drawn frames

 

The visual themes in this project will be networks within nature, creating links between distorted complex structures and microorganisms. The representation above is a CT scan. Part of the storyboard I have constructed for the video visualises the interior of a human body being entangled with nature. Such imagery will be layered onto natural backgrounds.

Sustainable Fashion project - Weekly 2

 Speaking to a friend who is a fashion designer, I have had a first look at sustainability in fashion. A tricky but extremely current subject matter for a project to have. As it is such an issue, We will find an effective way to display this theme.

Issues we have discussed is how we will convey this. We settled on not making an advertisement but rather to approach the issue more interpretively.

Initial Ideas presentation

 

The two concepts above are designed to help me develop my visual language and understand what it is like adapting a third party story into my own visual storytelling. Ideas like the ones above which are rich with topics to research and have the potential to have a striking aesthetic. For a final result, I would like to develop storyboards of these as mocks for a short animation series of true stories.

Diablo

Story

The true story of a group of housemates preparing to travel around Africa. Before they left, they had to decide what to do with their pet rabbit, Diablo. Tommy decides to take charge. He cooked the rabbit in a stew and fed it to his friends the day before they embarked. As they have all eaten Diablo (Devil in Spanish), each had their own near fatal accident.

Limitations

The contact I had who was going to share this story is being reluctant to give me the story out of wanting to use it in his own writing. I will attempt to negotiate and get further detail to the story, or make an adaptation of it if I choose to take this route.

Pop

Story

Have heard a number of short stories from my great Grandfather. He was in the secret service and has shared fascinating stories. None of which are to great detail but have given me an idea as to what it was like. His missions included jungle warfare and the Mafia. These would make for great animated shorts.

Limitations

Some of the stories are sensitive to not be shared with the public. Furthermore the details are could be gruesome. Asking for further detail into such stories could be a sensitive topic.

I have had the idea to do a callout on social media platforms for people to send in other true stories which would translate into good animated shorts. From these storyboards and possibly animations could be made.

The moodboard represents some of the research I am interested in. when it comes to musical narrative. The unpaid commission I am working on is for a Psytrance music video. The music is extremely fast and high energy. The song, named High is a representation of the effects on the brain during lockdown mixed with a feverish desire for a party. I have researched types of fungus, namely Cordycepts which are able to take over parts of insects' brains, control them and hatch out of their heads. I thought this aesthetic could be used well in a chaotic music video.

So far I have experimented with real photography of nature to create an idea of the networks fungus create. To link this to human activity, I have worked on a hand drawn representation of a CT scan. Using bright colours one might see in a fungus, I am beginning to develop the aesthetic for the upcoming 2.30 minute animation.

Stills from this project have already been framed and exhibited at the It's Going to Happen art show, London curated by Alice Crayson, which provided the work with good exposure.

This group project has the intention of using a number of mediums. Looking at fashion sustainability allows for many different creative decisions and aesthetics. We are hoping to blend music production, sustainable fashion, animation and projection mapping in our project. By looking at these allows for collaboration with non animation based creatives such as fashion designers and music producers. We will also have opportunity to engage in live briefs for clients. We are taking into consideration getting shows for catwalk runs, live dance performances and installation.

It is important for me to keep my mind open, new opportunities could arise throughout the year so it is important to select the right ones and not overload myself with work. Moving forward, I will make a Gantt chart and find an effective way to balance and present different projects within this module.

Extra research

 

Aphex Twin, Crush

Weirdcore portfolio

Brian Welch, Beats

Weirdcore works closely with artists to create live performance visuals. This includes classic a/v style visuals (futuristic coding and geometry). The programming is tightly timed to the music.

Weirdcore also creates dynamic 3D environments which constantly shift in form and subject. Brian Welch's film Beats contains a segment of Weirdcore's work. It shows rapidly changing frames of microscopic cellular structures.

His work on live performances has also included live distortion of people in the audience accompanied with diverse, chaotic material.

Hamill Industris portfolio

Hamill Industries and Floating points have been affiliated through their audiovisual works. Hamill Industries works using new technology and ideas to create diverse multimodal installation pieces. They often push the boundaries on how creative outcomes can look and feel.

Videos such as Anasickmodular represent sound reactant visuals which have diverse colour and form, the likes of which are often projected at live performances. Below are pictures from Anasickmodular and Floating Points' live performance at Printworks club, London.

The visuals represented are often a/v based. A practice which I will look into in the future. As it uses non cinematic practice to inform how it looks, it is somewhat irrelevant to what I am exploring except for a contradictory approach. However the topic greatly interests me and will further my knowledge of multimodality in music.

Projection mapping report

 Exhibit A

using Skream’s electronic track Settled can test how we can perceive visuals in combination with clear leitmotifs. As it progresses, the repetition of chords resonate progressively with each reiteration (much like the motifs within Jaws and Hereditary). I believe that combining timed visual cues boost the stimulus by providing more information.

Visuals are also performed live. Much like the meticulous construction of a motion picture, live shows will be choreographed and played in a specific manner. This means that visuals can be tailored accordingly when for example a climactic moment arises within a performance. This comes in many manifestations. The work of a/v practitioners will assist in defining the methods behind adding emphasis to music. Whether the individual’s work takes influence from film or not, there will be correlation between the methods used by artists such as Weirdcore and filmic theory.

Exhibit A is designed to correlate to perfectly timed visuals. Although the methods used to achieve this are different between my work and the visual work of artists like Bretschneider, the desired effect is the same and has links to cinematic theory. This effect is to assign geometric shapes to specific notes within the soundtrack while also creating a sense of anticipation within my hypothetical audience The short clip contains the first of my projection mapping experiments. To reiterate, I have used Skream’s Settled as my soundtrack because it represents a climactic buildup of leitmotifs. The viewer can see that each chord sounded comes in conjunction with a window on the model house lighting up.

Chion comments on the “ear’s temporal threshold”: That the ear brain system is only able to process auditory/visual information in sections. The information is “gestalt”, meaning that although they are essentially different entities, we as viewers perceive them together, which is how a practitioner would give these elements “added value” (the statement Chion uses when a stimulus is added to another one to boost cognitive engagement from an audience). Exhibit A represents this. The first leitmotif contains static sounds and images manifesting in the piece, as the scene evolves the sound and image begin to pulse and change colour. (the placement of the windows lighting up moves up and down with the musical chords to assist the audience in connecting the two). Each leitmotif essentially presents the audience with something new. The first iteration is processed as a section of a few seconds of sound and animation. The second changes slightly to give a sense of progression and anticipation. Therefore, the first test is a starter into adding temporality to the music and proves that “synchronous sound does impose a sense of succession”. 

Despite my personal findings and first hand realisation of this statement, the methods of creating such an effect are usually achieved through different means by audiovisual practitioners. Where I had created these visuals by manually selecting where and when to incorporate them, there is programming which will automate this according to the type of sound playing. Synthesiser based musician, Floating Points works with Hamill Industries to create audiovisual experience. One of their latest pieces, Anasickmodular represents the complex ways sound can be manipulated into image. In a commentary on this music video, Hamill Industries comment on their methods, saying “The Floating Points show is based on the visualization of sound through the manipulation of analog technologies. It will combine light, cameras, and visuals activated by sound waves, which in turn are modulated, creating image and sound loops”. The comparisons between Anasickmodular and exhibit A lie essentially in what each is achieving (visuals synchronised with music). However, Hamill industries devalues the ways in which to reach this because technology allows us to create the effect with the precision of a machine. Therefore, doing this manually could be seen as an obsolete practice, but still shows how image temporalisation can be applied to film and musical performance.

Exhibit B

The proceeding clips take image temporality into more broad terms to gauge if filmic theory can be useful further to what is discussed in Exhibit A. A key concept from Chion is that “sound vectorises or dramatises shots, orienting them toward a future, a goal, and creation of a feeling of imminence”, and that the shot is “going somewhere and it is oriented in time”. Skream’s song develops from the leitmotifs previously described to a crescendo, leading to the climax of the song. This section features less distinguishable notes but rather an ominous fade together of electronic tones. The visuals in turn present more abstract animation of sound by also fading frames in and out slowly. Without the sound, the moving image would have no meaning, but by applying it with the music. The viewer can see how the image has endowed the sound with a further sense of anticipation. This represents an evolved idea of how Chion’s theories are relevant to musical practice. Although the visuals are not timed with specific points in the music, they do give the music this sense of “added value” because the act of processing them simultaneously creates the desired effect of the future of the shot through the crescendo of the music and the slow progression shown in Exhibit B. This relates with the aforementioned Square Cube exhibit at the Electronic exhibition. Although I had first observed the piece in conjunction with untimed, unrelated music, the sound had still created a progressive sense, meaning that even visuals paired with separate auditory information can still be paired through the individual as a processing centre.

Exhibit C

The final test moves away from the ideals of a/v practice by using more organic forms to accompany music rather than non-narrative based visuals: “Movement of characters or objects, movement of smoke or light” Chion states that use of such material gives the image itself the sense of succession which would usually require the assistance of music. “Here, sound’s temporality combines with the temporality already present in the image. The two may move in concert or slightly at odds with each other, in the same manner as two instruments playing simultaneously.” Exhibit C represents this, initially demonstrating the movement of ink through water being introduced by the climax of the tune. The simple act of the animation running on the same timeline therefore creates temporality without the precise timing previously mentioned.

A similar practitioner to Hamill studios is Weirdcore, who creates bespoke live sensory experiences for many performances. Perhaps his most prestigious collaboration is with Aphex Twin (another electronic artist whose work was exhibited at the Design museum). One such collaboration is Collapse (2018), here the audience is presented with an amalgamation of visual techniques including a/v. However in this instance my focus is on the organic material used as the subject matter. We see a virtual camera zooming through an architectural environment based on pictures the artist took of Aphex Twin’s home town in Cornwall. Much like the ink movement represented in Exhibit C, the movement through space gives the scene forward momentum. Imagining the progression of the audiovisual gives a good impression as to how Chion’s theory of temporaliastion works. Similar to how the camera hurtles through space, showing different cobbled streets as the music plays, My demonstration shows the motion the ink takes in water as an accompaniment to the music playing. I applied other organic animation to further demonstrate this. A rotoscoped face with mouth moving was applied over the ink animation with imprecise timing. Despite this, the mouth shapes move in time with the music creating further links between the separate stimuli.

Aphex Twin, Crush

Application to electronic music performance

Application to electronic music performance 

My exploration into electronic music could provide answers as to our perception of music. Instead of tailoring harmonies to accompany the visual, the music will essentially come first. This section will continue to use Chion’s theory in correlation with other audiovisual artists while drawing comparisons to my practice.

The transition of theory from film to live performance involves reverting which stimulus is prominent. Although this ratio differs between every project, the narrative within a motion picture will often take the spotlight, with items such as sound being applied later on. Chion states that the soundtrack can “directly express its participation in the feeling of a scene, by taking on the scene’s rhythm, tone and phrasing”. Our ability to process the visual and other elements comes from the speed of our cognition and provides explanation as to why we will latch onto what is happening on screen. The Psychology of Sound blog evidences that “The visceral, emotional reactions that strong visuals can evoke are even quicker for our brains to process than emotion-neutral visuals — 13 milliseconds on average. Responding to sound is only slightly slower, at 146 milliseconds on average” this essentially explains why a viewer would latch onto visual stimulus quicker, while accompanying it with the other senses in a proceeding fashion. When bespoke soundtracks are made for a film, they are usually blended harmoniously, allowing for the music to take the back seat while directly affecting the way we respond to them. Although “we persist in ignoring how the soundtrack has modified perception”. Here one can see how even without awareness of audiovisual symbiosis, the brain is able to process miniscule details to accompany them seamlessly. 

A reiterating theme within electronic music naturally comes to sound reacting visuals, meaning that different visual stimuli will appear determined by what bass, lows and highs are playing. This is important because it connects to Chion’s first theory of temporal animation in film: “to varying degrees, sound renders the perception of time in the image as exact, detailed, immediate”. This highlights the simplistic method of applying sound to fit exactly to what is happening on screen. The equivalent in the case of music performance would be synchronous visual stimulus accompanying aspects of the music. Many a/v artists use geometric shapes to match with the music and react in different ways. Unlike in cinema, this form of visuals does not generally contain a narrative. As electronic/visual artist, Frank Bretschneider states “I use more or less abstract forms – simple shapes, geometrical patterns – rather in an animation style than cinematic. Like the status lights on electronic sound equipment, representing the parameters of a music piece: flashing and moving bars, dots, lines and numbers.” Even without the presence of a story or narrative, comparisons can be made between this form of a/v and cinema because apparitions on screen accompanied with a specific sound provide a complimenting relationship between the audiovisual. I recreated this within my own work to personally explore how the synched sound works with visuals and the effect it has when processing them together. Bretschneider elaborates to say “I love to have the visuals connected to the music, synchronized and tight. In the best case, it should represent the sound on the visual level.” This has been coined in my work, wherein I have created a 3D installation piece. Using projection mapping, I have represented a number of Chion’s theories in correlation with audiovisual practitioners’ work. I chose projection mapping in order to have multiple plains and dimensions to animate on. I felt this would best represent a three dimensional performance space and mimic a live electronic music performance on a small scale. Therefore I have built a scaled down 3D house to represent the type of installation an audience might expect at a live performance.

Completed Gantt chart

WWW

-Time management evenly balanced, fairly consistent work throughout

-Practical and written work spread evenly

-The detail has given me a better idea of the jobs required. This will make it easier to make Gantt charts in the future.

EBI

-Practical work should have been prioritised earlier on

-Could have been achieved by predicting jobs which need to be done