Weekly

 

Production Week 5

 https://youtube.com/shorts/OzzEaT8BHiY?feature=share

UPDATED ANIMATIC

 

Updated Scene

 

Production Week 4

 

Scene 11/12 Reference Footage

Scene 11/12

Scene 20

Scene 21-25

Scene 26-33

Production Week 3

 

 

Scene 5

An animation test to make background work less repetitive, and because the background for scene 5 is simple. It works well as an introductory shot of the character following establishing shots of the surroundings, but will probably need colour work later on.

Scene 8

Perspective tunnel shot - Will definitely need some textural work done. Atmospheric sound is incorporated into this scene so drops of water and echoes will be worked on.

Scene 10

This makes up the establishing shot of the DJ characters. The layers are also organised for a far more busy scene at the end of the film. The room will be full of people, rotoscoped from reference footage I have collected.

Production week 2

 

Reference Images

Scene 6/7

The refined backgrounds show a zooming shot. No sound is present so the scene remains grey.

A portrait mid shot for panning down to the sewers.

Scene 16

Wide shot of buildings. Windows marked out for a composite of sound/colour.

Scene 26

Another sound rich shot. Character to be composited in

Scene 2

More establishing shots for compositing.

Propoganda for opening scenes

This use of vandalism and word play is one of the concepts I have considered in showing the change of societal narrative in the film. The first image represents state control. The second, a radical changing of the message which is communicated.

Another word play is:

24 Hour Surveillance  -  24 Hour Subservience

Since there is no spoken word in the film, items like propaganda will aid the message of the film, establishing the dystopia in the film and the changed attitudes at the end. I was inspired by Adam Curtis v Massive Attack as it too uses big text with haunting undertones.

The second video has text that reads:

 ‘NOW IT’S YOUR WORLD [...] YOU ARE AT THE CENTRE OF EVERYTHING [...] IN THE PAST POLITICIANS WANTED TO CHANGE THINGS [...] INCLUDING YOU [...] THE NEW SYSTEM LISTENS TO YOU, OBSERVES YOU, UNDERSTANDS YOU, AND GIVES YOU WHAT IT KNOWS YOU WANT’

This is a good example of some dangerous concepts of state control and societal compliance that are valuable to feed into the narrative. Text is a good way to explore this.

MASSIVE ATTACK.IE (2016) Behind the Scenes of Massive Attack v Adam Curtis [online video] Available at: https://www.youtube.com/watch?v=3e3I1mLc3RM [Accessed 19/04/2022]

Manchester International Festival (2013) Massive Attack v Adam Curtis [online video] Available at: https://www.youtube.com/watch?v=yv_S8GdylEA  [Accessed 19/04/2022]

Production Week 1

 Scene 1

This took a really long time to animate. It is rotoscoped from reference footage for a good impression of changing perspective. Colouring and window drawing would be so time consuming that the scene may have to stay simple as I have many more scenes to work on. 

As the first scene of the film I am considering a text overlay to introduce the film and add a bit of dynamism to the shot. Arguably it does work as an underdeveloped scene and may not need extensive detail. I will readdress this situation when editing it with other scenes. Upon completion each scene is being composited in After Effects.

I am paying constant attention to the animatic and storyline. Any details I can add to reinforce the state controlled nature of the film, the ensuing protest and the audiovisual elements within will be applied.

LAUMAAN 705 Goals

 This Trimester my goal is to complete Big Smoke to the best standard it can be so it is festival ready. It is important to me that I can showcase it at appropriate events and know timings for promotional work.

I will achieve this by:

- Referring to storyboard and developing how to reflect research into the narrative.

- Refining my animated work both with scheduling and practice.

- Developing sound animations

- Festivals and promotion packages.

- Developing a document for portfolio of the making of Big Smoke (an extension of the pitch bible and pre production output).

The production schedule is laid out but will likely need ramifications as I progress. For the next 5 weeks out of 14 (including Easter break) I will be mainly working on backgrounds alongside some animations. These will be ready for compositing and characters to be animated in.

Final Testing

 A few images from the latest composite:

Back layer for windows to light up to the music.

Some more established background shots

Here, I have done an exercise where I listened to the audio track made by my house mate made. Looking at background and character path, I have imagined the field of sound, much like Futurist paintings, and imagined a path for it to take. Using onion skin, this is a representation of the sound path. I have not added it to my final composite as I want it to be more detailed and colourful (an exercise I will tackle in the coming weeks). 

This was a good exercise for thinking of the sound as a field and translating it into animation. This will get my practice in line for the research I am doing by translating sound and feeling into animation.

This is the final composite for one of the final scenes of the film. Comparing it to my other composite, it was far more complex and time consuming. The scope it has provided me however has been extremely useful. I have mocked up what I feel to be an accurate timetable of the next 14 weeks. It involves a strict background drawing session up until the 10th May where I get detailed line and colour work done. This is followed by animation leading up until July, taking 3 days for complicated scenes and 1 for simpler ones. By learning tricks and shortcuts on my chosen programmes, the production will be far more streamlined and manageable.

Link to schedule:

https://docs.google.com/spreadsheets/d/1ZCVRSeDfE6sV0UqSsJBcHP30WY51Lt92/edit#gid=756394758

This is another venture for the coming weeks. Expressions are a tool to use when creating audiovisual synthesis. This is a test pairing wave warp effect to the audio levels. It needs some work but has exciting prospects for mixes of colour and manipulation of sound and image. I hope to learn more about expressions soon.

Composite 2

 The first completed composite demonstrated the first instances of synaesthesia in the film. This was to get an initial idea on how to represent the sound while researching the multimodal field. Now that has been realised, the final stint of work before completing the module is to create a more accomplished scene where the sound has built and the backgrounds and characters are more diverse. 

This is in progress but parts have been realised. Once this scene is completed, I will map out a more accurate timetable for the final process of production. This is for good timekeeping and discovering how long these processes may take in the coming months.

Run Cycle

The completed run cycle works well for composite. The layers are all separate so I can manipulate colour in line to background ongoings. The shadow moves in a smoke-like way which I am hoping will create match fluid sounds within the audio track. A few more expressions and running styles may be added to the loop further down the line to add dynamism to the cycle.

Scene 34/35/36 No Colour

At this point, this scene has many problems but transfers the atmosphere and actions in the scene. The layers aren't in order so I need to go back and edit the composite. In this process I have learned shortcuts between Adobe programmes and TV Paint. The process was a good learning curb to streamlining production.

I can update my schedule with a far more accurate idea of timings and feel much more confident that the film will be completed on time. As I will have completed 2 scenes for this trimester, both with different levels of sound representation, I am now able to map out the next 14 weeks of animation and have really come to realise the scope of the work through this module.

The updated scene with correct compositing will be up soon...

CONTEXTUAL REVIEW 2

As an animator in a changing world, I have been looking into influential filmmaking through a commentary on modern society. I have been using my knowledge of storytelling and multimedia to evoke true experience from my audience, finding ways to link my interest in music and film with a wider socio-political background.

Multimodal Experience

Multimodal theory has been the foundation of my research. Lovers of film will undoubtedly recognise the power of harnessing the sense modalities in a way that resonates with audiences, whether by using music to boost emotion, or engaging in the many other cognitive crossovers that are possible within creative processes. In the study of The Relative Importance of Local and Global Structures in Music Perception, the commentary is made that 'One of the main reasons leading human beings of Western culture to develop musical activities is the expressive power of music. The content of this expressivity is probably extremely large, rendering the domain of music expression and emotion difficult to address with scientific methods' (Tillmann and Bigand, 2004, p.211). This provides consideration that in the field of cognitive science, which is expansive to the extent that is not fully understood to this day, using methods of multimodality, artistry and human reaction are apt approaches to gauging cognitive engagement with a piece of media. All of which are idiosyncratic to the individual. A standalone piece can resonate in diverse ways with different viewers. It is interesting to think how external forces of circumstance or other sensory content could create a completely different affect to your neighbour.

The MIT press, Experience: Culture, Cognition and the Common Sense supports the theory that artistic practice can be valuable for studies into perception. Studies in the early 20th Century lead to ideas about cognition through art that had not been considered before. The opening of the book theorises that 'art demonstrates to science the broad political and cultural context forming what sense we make of experience [...] art is not the semiotic conveyance of a fixed message-It is a range of marks, indices, rituals, and materials that humans make, arguably uniquely, to simulate perception. We do these things to draw upon prior layers of experience, in order to produce, paradoxically, the genuinely new'. This is an exciting prospect as it acknowledges the power of art to innovate ideas and change the world through the experiences it provides. However it still poses the question of how to conduct practice in a way that communicates ideas experientially. My studies will attempt to harness this through future case studies and qualitative research. 

Sound and image accompaniment as aforementioned are the key sensory crossovers I will explore. Michel Chion is an influence because his theories into sound design for film indisputably boost the idea that sound and image are malleable (and modern) mediums which can be used to boost or distort perception. The Experience book contains ideas about temporality, causality and projection, theorising that 'Cognition implies a degree of plasticity for human psychophysical processes; in this sense, it can be as active and adaptable as consciousness itself'. With Chion, you can see this cognitive harnessing in action through his case studies. The detailed descriptions have commonality with the experiential studies because they both support the fact that minimal visual cues can invite projection, which is perceiving a temporal procession of events.

Italian Futurism

The Futurists conducted studies into the form of non tangible entities, combining motion, sound, kinetic energy and political ideologies. F.T. Marinetti dreamed up the concept following a car crash he was involved in. The translation of human experience and inevitable industrial prowess is evident in the art, which largely depicted motion and mechanical shapes. This guided the direction of the style. Despite the stock Marinetti puts stock into the industry of the early 20th Century with word of 'snorting machines' and 'the rumbling of huge double decker trams', David Mather argues that among the visually changing world, new experiential qualities had to be processed in order to embrace modernity:

'Western notions of subjectivity were in crisis at the turn of the 20th Century, in response to experiential conditions of modernity. This crisis was not the product of new technology per se, despite coinciding with rapid technological invention. Rather, it comprised a set of perceptual and philosophical conundrums that, in effect, rendered the boundaries of individuality itself problematic.' (p.58)

The world was rapidly changing, which came hand in hand with unfamiliar sights, sounds and smells. Not only did the Futurists map the changing world around them by incorporating these sensorial qualities, they also tapped into the emphasis of inclusion of all fields of cognition, recognising that the resonance on their viewers would be boosted by exploring multimodal interpretations of a modern world.

'Speed had disrupted traditional ways of life, and the Futurists embraced it unflinchingly. Yet, this seductive narrative of Futurist enthralment with automobiles and other means of transportation measures only the most literal contours of their subject matter. Alongside references to kinetic apparatuses, their visual experiments moved steadily away from representing discrete objects and toward emphasising sensorial qualities - what it felt like to experience the rapid and unsettling forces of modernisation.' (p60)

This commentary adds context to the stylistic progress of the Futurists. As they developed ideas about the changing world, there was an evident yearning to exceed the boundaries of human perception. 'The Futurists' visual systems shifted from external observable phenomena to internal conditions of perception, and from there, to the psychophysical mediums that bridge external and internal processes'. By using the Plastic Arts, the Futurists were creating a commentary on History and the progression of the human race. Using sensory attributes helped to situate their work in the context of the world around them. Depictions of sound interweaved with the physicality of machines were ways of observing the phenomenon of the future.

We Futurist painters maintain that sounds, noises and smells are incorporated in the expression of lines, volumes and colours just as lines, volumes and colours are incorporated in the architecture of a musical work. Our canvases therefore express the plastic equivalent of the sounds, noises and smells found in theatres, music-halls, cinemas, brothels, railway stations, ports, garages, hospitals, workshops etc. etc.

Sound is one of the main sensory attributes I will explore in this dissertation. As I come to realise the philosophical usefulness of sound, the differences between the Futurist ideologies and their use of the multimodal are becoming apparent. Despite using their art to promote controversial ideas about industry, warfare and the dismissal of History, their uses of kinetic (or otherwise) energies were effective ways of communicating their message.

'The artificial amplitude of sound is one of the great inventions of the 20th century. Modernity may be defined as the coming of the human capacity to make inhuman noise. The great shock of the modern city and of the modern warfare that was in extrapolation were not so much the experiences of their disorientating energy and speed, as their sheer noisiness, the appalling, exhilarating, omnipresence of man-made or mechanical sound: of cars, sirens, gramophones, loudspeakers, cannons, airplanes and industrial machinery; all the dinning cacophony of the modern.' 

This commentary emphasises how the new 'cacophony' of sounds were as informative about the mood of the 20th Century as the visual was. It is no wonder that the Futurists became so enthralled by depicting the multimodal, experiential changes happening everywhere.

A great influence on the Futurist sensibility was the works of Philosopher, Friedrich Nietzsche. 'Nietzsche’s influence on Marinetti and the Futurists has been well documented by Giovanni Lista, Donald Marinelli and Günter Berghaus'. His influences on Freud also provide credibility to his research as thinking that is influential in a contemporary context. One of his observations brings relevance to the way I will use music in my explorative film, Big Smoke: 'For Nietzsche, after Schopenhauer, music was superior to all other arts since it did not represent a phenomenon, but rather the ‘world will’ itself. He regarded music as the non-representational bridge to the chaotic world of the unknown and to the creative impulse itself.' Thinking about the rise of Fascism, war, Industry and general modernisation of the world (the context for Futurist thinking' in relation to my project, parallels can be drawn. The state controlled world, use of propaganda and pollutive powers are present in both. The Futurists were embracing it as the inevitable world trajectory. I am using these ideals as a guideline to avoid by representing modernisation as a negative. 

The Futurist's Plastic Arts are valuable to my studies. I can use the stylistic attributes of sound in an affective way without supporting their political beliefs. In the present context of my film, the music can be used as the 'bridge' (using Nietzsche's beliefs)  to emphasise the global misconduct of Government present today. Mapping the synaesthesic movement of sound using colour, if harnessed correctly in my practice, can aid the positive messages within my film that creativity can be the epistemological guide to a better future.

Tillmann, B. and Bigand, E., 2004. The Relative Importance of Local and Global Structures in Music Perception. The Journal of Aesthetics and Art Criticism, [online] 62(2), p.211. Available at: <https://www.jstor.org/stable/1559204?casa_token=lrEEMwuM6lMAAAAA%3AxXgFbNz5p5ZZwyewh1anWgQhWba0B9OKIb-fOqWqy4vrU4jV8XTxvPRhkmR0gEI6b5RtAsUXpukTTaBNL2J99bxkQj5zGY7yy38vx27KWx-TtpsZqPo&seq=1#metadata_info_tab_contents> [Accessed 9 March 2022].

Situating the Practice

 My animated music video fits into a few areas of professional practice. Creating a venn diagram of these areas has helped me to categorise the different fields of interest.

The Diagram has helped me define and consider which studios may be in line with my work, and which festival entries are applicable. The Trimester has seen much more practical experimentation than networking skills (which were set out in my initial targets). Luckily, there is a contingency to this work. When the production module comes round, expanding on this should be a key part of my research to start pushing the film, and my skills, down the right avenues. 

Here are a few initial ideas:

Blink Ink:

Use current animation discourse to create short films for companies. They delve into music, fashion, general advertising and some feature films. I have done work experience with them in the past and they are a viable studio for me to work at.

Over Lockdown, they were creating animated content for musicians like Dua Lipa, utilising lack of human contact to create content for artists and creating fantastical worlds of music, art and colour.

Club Future Nostalgia Mixtape (2020) Directed by James Papper [Short Film]. London, Blink Ink.

International Music Video Underground:

Just one festival I found on Film Freeway (previously recommended for multiple animation awards). They have entries for short films, animated music videos and other awards relevant to my film's themes. Film Freeway will be a good resource in general.

A note about film festivals is that my entry cannot be made until the film is completed (in July) as many awards may have passed, I may have to wait to enter in the coming year, meaning I would have to keep the film under wraps. A benefit of this is that it allows ample time to create promotional material, which I have learned to a good level recently through event organising and a decent knowledge of brand identity and promotion.

Science-Fiction and politics:

Animation is a good way to make commentaries on the world. It can be exaggerated or simplified. Big Smoke highlights problems in the world at the moment of isolation, Government misconduct and increasing violations on privacy, identity and freedom. Finding films which exaggerate such themes (particularly in a dystopian style) can help me see stylistic features of a breaking world.

Orang Tan - Greenpeace

Greenpeace International (2018) Rang-tan: the story of dirty palm oil [online video]. Available at: https://www.youtube.com/watch?v=TQQXstNh45g&t=24s [accessed 21/03/2022]

Monster - Greenpeace

Greenpeace International (2020) There's a monster in my kitchen [online video]. Available at: https://www.youtube.com/watch?v=8rIjVDo_u8c&t=132s [accessed 21/03/2022]

Akira:

In an overpopulated world post World War 3, Akira is at the epitome of dystopian animation. With a higher budget, more time and a team, I would want Big Smoke to evoke much of the visuals within this film, the colours, character and backgrounds all evoke a beautifully represented dying world. 

Akira (1991) Directed by Katsuhiro Otomo [Feature Film] Japan: Toho Co., Ltd.

THE BEST TIL LAST....

Jordorowsky's Dune:

Jordorowsky's Dune (2013) Directed by Frank Pavich [Documentary] Snowfort Pictures

Veering away from Dystopia and more towards Sci-Fi, this documentary about the Arthaus film that never happened was fascinating, involving a show-stopping team (including Mick Jagger, Pink Floyd and Salvador Dali). Discourse about this film is ongoing. Storyboards and pre-production were completed in a book going for auction for millions. Somebody has bought this book and is creating a community through a crypto-currency called Spice. Aptly named for the fans of Dune. There is talk of an animated version of the film to be produced. It is a whimsical idea through all the rights involved in creating the film. However this is a bit of a future dream so I thought I'd include it in this post.

Testing Testing

 WAVE WARP

This After Effects tutorial looks at creating wave ripples and distortion. My idea was to culminate the layers I made in the style of Carlos Cruz Dies as background sounds. to create colourful instances of sound in the background, the panels of colour will distort more and more as the film progresses. 

So far, I have experimented with rotational and positional transformations to make the scene more dynamic. The wave warp effect will help create fluid-like motion for the sounds. 

Further to this, many of the assets in wave warp can make my visuals distort. Using expressions, I am currently seeing if this warp can react to music and create the waveform I am looking for to visually represent my sound. This combined with vibrant, complimentary colours could be an effective way to represent some of my sounds.

A screenshot of the style wave warp could add to the physichromie animations. Expressions could create changes in the ripples in time to the music.

GRADIENT FILL

Gradient Fill will help me set the sounds into the picture. If I feel the opacity needs to come in round the edges, this tool will allow me to do so, thus setting elements of the composite more into the scene.

TEST SCENE

This composite is essentially the base layer of character animation. This will be the initial stages of all the animation of the film, with parallax's, sound animations, colour and lighting edited in After Effects. This forms the structure of production within the entire film. 

FINAL SCENE

This is a finally realised scene with nought but minor adjustments to make. Exporting out of After Effects is important to ensure timings are correct. Some of the transformational movements need to be more subtle in the scene as well. As I have changeable foregrounds and backgrounds to convey a sense of space in the scene, I need to ensure that the movements of camera etc. are subtle and believable. Most of the shots work in a directional way to drive the narrative. The Close up shot which zooms could be less fast paced as it currently looks slightly unnatural and distracts the flow of the scene. Now that I have timings of making a scene more realised, I can update my production schedule to more accurately represent the scope of the film's production and how it will be achieved on time. There is still room to edit this scene as I consider it about 95% finished so there is no commitment to assigning this scene as the finished look for the film. More is to be realised..

The updated scene includes a the first instances of visual sound in the film. Using After Effects, I could position the sounds wherever and make them look like a part of the scene. There is scope to change the look of the sound (if not in this scene, definitely as the music and narrative builds). I must create a field of sound which reacts in ways that are well considered to further reflect the kinetic force and trajectory that is present in Futurist painting.

With this scene complete (which was a good choice for the testing process of this module as I began to discover how to depict sound), it would be good to now look into a scene later on in the narrative. After the final crit for 703 I have decided that animating another scene further on in the narrative will wrap up the practical outcomes for the module. I will choose a scene with rich sound and diverse animation to test and explore further the potential for the progression of multimodal-ness in the film.

Italian Futurism Study

 Practical Outcomes 

Futurism is becoming prevalent in the dissertation module. Their depictions of motion using force lines, and visualisation of multiple sensory modalities are exactly what I'm looking for in my studies into making experiential animation. As the research is an overarching the entire concept for how my film is developing it is appropriate to demonstrate this in the 703 blog. 

Example of force Lines in Italian Futurism: 

The Hand of the Violinist (The Rhythms of the Bow) - Giacomo Balla

Balla, G. (1912) The Hand of the Violinist (The Rhythms of the Bow). [Oil on Canvas]. Estorick Collection of Modern Italian Art, London

 Context  

'the recognition of the artist embracing modernity and acting as the creator of the future.'

If we leave aside their radically different literary form, both statements, full of youthfully buoyant spirits and distrustful of the past as an artistic source, stressed two chief points: the artist as creative seer and guide to the future, located at the heart of the most vital modern activity, and the necessity to recover a pure, untrammelled sensibility to express the novel values and experience of the changing world.'

These new realities manifested themselves in what he named "les unanimes," or vast collective sentiments, which created a flux of uncanny physical and spiritual relationships voiding classic concepts of space and time, thus overcoming the isolation of Leibnitz' monads. 

In Leibniz's system of metaphysics, monads are basic substances that make up the universe but lack spatial extension and hence are immaterial. Each monad is a unique, indestructible, dynamic, soullike entity whose properties are a function of its perceptions and appetites

 David Mather Essay 

'In early 1914, the Paris based Italian Futurist painter Gino Severini coined the term Plastic analogies to describe a method for representing modern perceptual experience.. He did not intend these analogies to correspond with everyday perception, or with its conventional representations. Rather, he meant that an artist's knowledge, particularly firsthand, sensory knowledge, could be translated into any of the so-called plastic arts, so the viewer interacting with works of art could access experiential content directly from its forms.' Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.57.

Plastic Analogies - key point for argument

Severini believed that plastic analogies worked through three fundamental elements; an experiential subject matter, a non-naturalistic method of transcribing the source through diverse modalities; and a viewer who would interpret the work for its complex interconnected cognitive meanings. 

Methodology for creating musical visuals for the film

The plastic manner was phrase Severini used prior to 1914 to describe phenomena which cannot be recognised by just one sense modality or symbolic form. The viewer needn't know the subject matter to recognise the subject or the sensory richness. The experience is still arguably translated in abstract form.

Severini, G (1914) Sea Dancer [Oil on Canvas]. Guggenheim, New York

'An analogy between a geographic feature and a moving human body. Yet these phenomena are not distinguishable in the work, and the artist even indicated that this particular composite accurately represented neither, but evoked both.' Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.57.

'While the majority of Futurist paintings portray images of machinery, Severini instead chose the dancer. It is fair to say that no other painter of the movement shared his enthusiasm for this type of subject.' 

Haidinger, M., 2014. "Gino Severini's Dancers and His Theatrical Milieu," Art Journal: Vol., pp.13

'Tonal modulations of semiopaque shapes which, for a spectator, should connote expenditures of kinetic force analogous to both.' Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.57.

Visual Kinetic Approach

'Western notions of subjectivity were in crisis at the turn of the 20th Century, in response to experiential conditions of modernity. This crisis was not the product of new technology per se, despite coinciding with rapid technological invention. Rather, it comprised a set of perceptual and philosophical conundrums that, in effect, rendered the boundaries of individuality itself problematic.' Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.58.

A question that could be raised in the end of the essay is whether, in the new context and knowledge of global preservation, plastic analogies can drive towards a sustainable future. Using the forces (visual, kinetic or otherwise) and some of the ideas of Futurism, could more humanitarian ideals be brought to the forefront to drive a better future rather than the technological advancement and industry of the early 20th Century?

'The moving object is unrecognisable within a field of superimposed glints and sonic reverberations that shudder across the frame'. Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.61.

'Visual patterns highlight the visual and auditory sensations associated with vehicles observed from the side of the road'. Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.61.

With ideas of Chronophotography, motion is brought into the Futurist paintings. Temporality is compressed into one abstract image. Time is taken out of the equation to make a sequential, but still, image.

'Apprehending the underlying truths of experience' Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.61.

Somatosensory

relating to or denoting a sensation (such as pressure, pain, or warmth) which can occur anywhere in the body, in contrast to one localised at a sense organ (such as sight, balance, or taste).

Sensorimotor
Sensory-motor coupling is the coupling or integration of the sensory system and motor system. Sensorimotor integration is not a static process. For a given stimulus, there is no one single motor command.

'The Futurists' visual systems shifted from external observable phenomena to internal conditions of perception, andm from there, to the psychophysical mediums that bridge external and internal processes'. Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.63.

'Implicit within the chromatic geometry is the play between straight and curved edges, as well as the adjacent pairing of complimentary hues: red-green, orange-blue, and violet-yellow'. Mather Uchill, J., 2016. Experience : Culture, Cognition and the Common Sense. MIT Press, p.63.

CONTEXTUAL REVIEW

My exploration will delve into cognitive responses to sound and music with a focus on gestural movements and animations. For a controlled contextual review, I will first examine instances of images triggering experience, followed by sound. By looking at these in their own respective modalities, connections will be drawn between the two. 

A principle within the MIT press of Experience: Culture, Cognition and the Common Sense frequently refers to projection; a causal event wherein a viewer/listener/experiencer can foresee events which happen next. The way individuals process information is all unique, but somewhat governed by the fact that ‘our experience of coherence and meaning is produced by activities of modelling - processes of sifting impressions, cascades of calculations, and attributions of significance carved from determinations of causality, which are in turn embedded in and generated by environments we have come to know’.

In a time where artistic multimodality may have more frequently manifested in still images, some interesting thoughts and responses were emerging. In the early 1900’s, Italian Futurist painter Gino Severini was depicting experience based on his own perception of particular events. Plastic analogies (a term Severini coined) were his methodology in mapping out ‘imagistic composites that combined two or more symbolic forms into a single image’(p57). The fact that movement, motion and even sound were being depicted in still images from 1914 is astounding and represents the importance people place on exploring cognition and how artistic practices are an outlet to record, analyse and discover.

In 1943, responses of using artistry as a way to gauge cognition and experience were emerging in the form of simple animations. Fritz Heider and Marianne Simmel experimented with endowing simple shapes with motion: “The projected image shows an outline of a rectangle with an opening on its right side; near this opening the shape of a triangle arrives. Soon a smaller triangle and a circular disk also enter the frame from above. The rectangle remains static, but the disk and triangles become quite literally animated, and each moves in a manner that comes to seem characteristic. That is, they are given movements that model different characters.’(p13) The aim was to research how viewers may perceive these shapes with attributes which go beyond their two dimensional limitations. A spectrum of titles to the research emerged, from ‘Perceptual of Geometrical Figures to Warring Triangles’.(p14) These titles alone aptly represent what the researchers were aiming to achieve. Representing a film ‘devoid of content’ yielding a range of fully fledged stories. The essay I refer to comes from the MIT press book, Experience, and the Heider-Simmel film is noted to have stock in contemporary studies of neuroscience, psychology, and the ‘aesthetics of ethics’(p14). The knowledge that artistic practices can be as informative to the studies of cognition and experience as scientific methods can, alongside the epistemology that 'Cognition implies a degree of plasticity for human psychophysical processes; in this sense, it can be as active and adaptable as consciousness itself'(p8) provides a good framework for my investigation as I elaborate into my own fusion of character animation and abstract representations of sound to stimulate genuine responses from viewers from a variety of sensory modalities. Furthermore, this research continues to be relevant to modern cognitive studies at MIT, with Josh Tenenbaum using the film within studies of Visual and Environmental Studies in 2014. ‘Modelling is a practice common to both artists and scientists, and as a customary activity of humans in general; our modelling behaviours distinguish us as a species. We model ideas, we model clay, we model with pictures, texts, clothing. What were Fritz Heider and Marianne Simmel modelling when they made their enduring film? The short answer is projection. But on a larger scale, they were modelling the complexity in the human psyche’. Here we see this film within a modern context; exploring the malleable nature of cognition as an artistic tool, these films and studies become a heuristic approach to cognitive science.

There are even links between the experience of film and the principles of animation. The 12 principles of animation have much correlation with how one may evoke genuine experience in a viewer.

Look at multimodal approaches THROUGH THE LENS of Italian Futurism and Plastic Analogies

Reading - EXPERIENCE MIT Press

Key Words:

Projection - Cognitive ability to move temporally and create understandings of what is to come.

Kinetic - of or relating to the motion of material bodies and the forces and energy associated therewith

heuristic - allowing for someone to discover someting for themselves

'Cognition implies a degree of plasticity for human psychophysical processes; in this sense, it can be as active and adaptable as consciousness itself.' p8

'This book exists to complicate that model. [nervous system as a series of inputs and outputs being sent to the central processor or brain] Experience is our heuristic, and we intend both to provoke experiences in these pages, and to examine how culture and technology deeply weave the texture of human consciousness.' p15

'David Mather explores in the practice of Italian Futurists, bridging the cosmos and our own afferent/efferent nerve signals [input/output]; the Berliners' abstractions invited projection, proving that humans could construct meaning and experience from minimal perceptual cues.' p22

'Art demonstrates to science the broad political and cultural context forming what sense we make of experience' p26

Adorno's model replaces notions of  art as representation or 'expression'. Art is not the semiotic conveyance of a fixed message-It is a range of marks, indices, rituals, and materials that humans make, arguably uniquely, to simulate perception. We do these things to draw upon prior layers of experience, in order to produce, paradoxically, the genuinely new'. p26

Gives relevance to the artistic research of experience in the modern world without extensive scientific data collection. p28

Starting to map kinetic energy on a flat surface using artistic mediums. p61

'