An experiment in interrupting the congruity of audio-visual relationships
override is a series of two experimental works for mobile phone investigating creative ways in which the experience of audio-visual relationships may be interrupted to facilitate a more considered encounter with the senses. The aim of this project was to gain a more visceral understanding of how sound might function in combination with vision, and how sensing and sense-making might relate to each other. In other words, I was exploring an embodied understanding of Chion’s (1994) notion of synchresis. Somewhat led by Niall Moody’s (2009) thesis that the motion inherent in sight and sound acts as a connecting device between them, I set out to create an environment in which the congruity between the senses would be interrupted.
override developed into two audio-visual works to be experienced on a double-decker bus with the help of a mobile phone and noise-cancelling headphones. Both pieces aim to create discrepancy between seeing and hearing with the help of the movement felt whilst riding on a bus. In both cases the visual aspect of the film shows a journey made on a similar bus, whilst the sonic aspects and their relationship to the visual and kinetic information differ.
This article will begin by providing a project outline of override, as an experience of making at the intersections of hearing, seeing and kinetic sensations. It then discusses several topics relevant to its making: the relationships between synchresis and motion, virtual and real experiences, the role of listening in the development of the project and the journey from sensing to sense-making, explored from the perspective of creating override. One key point that emerged from this project was the realisation that conceptualisations – including preconceptions – readily insert themselves into sensorial experiences. Attempting to interrupt sensorial congruity therefore offers itself as an opportunity to question the many assumptions that are part of the sense-making process underlying our creative process.
Barthes, Roland (1977). Image music text. (S. Heath, Trans.). London: Harper Collins
Benford, S., & Giannachi, G. (2011). Performing Mixed Reality. Cambridge, MA: MIT Press.
Bourriaud, N. (2002). Relational Aesthetics. (S. Pleasance, & F. Woods, Trans.). Dijon: Les Presses du Réel.
Brown, E. (2006). Folio and Four Systems. CD Liner Notes. Tzadik Records.
Chion, M. (1994). Audio-Vision: Sound on Screen. (C. Gorbman, Trans.). New York: Columbia University Press.
Coughlan, S. (2011). ‘Fake’ paintings trick viewers in brain scan test. BBC News. Retrieved from: https://www.bbc.co.uk/news/education-16032234
Drever, J. (2017). The Case for Auraldiversity. Acoustic Regulations and Practice: The Hand Dryer Noise Story. Proceedings of the 24th International Congress on Sound and Vibration 2017 (ICSV 24), 5, 3576-3581.
Eisenstein, S. (2010). Towards a Theory of Montage. Sergei Eisenstein Selected Works, Volume 2. London: I.B. Tauris & Co.
Farinati, L., & Firth, C. (2018). The Force of Listening. Doormats No 6. Berlin: Errant Bodies.
Fodor, Jerry A. (1983). Modularity of Mind: An Essay on Faculty Psychology. Cambridge, MA: MIT Press.
Garrelfs, I. (2015). From inputs to outputs: an investigation of process in sound art practice (doctoral dissertation). University of the Arts London, UK. Retrieved from: http://ualresearchonline.arts.ac.uk/8342
Garrelfs, I. (2016a). From Conceptual Blending to Procedural Blending: Applying a Model of Cognition to Process in Sound Art Practice. In: Denham, S., & Punt, M. (Eds.), Off the Lip: Transdisciplinary Approaches to Cognitive Innovation. Conference Proceedings 2015. Plymouth: TT OA Papers, 71-88. Retrieved from: http://hdl.handle.net/10026.1/4271
Garrelfs, I. (2016b). Traces in/of/with sound: an artist’s experience of audio-visual space. Paper presented at Sonic Environments/NIME (New Interfaces for Musical Expression), Brisbane, Australia.
Grimshaw, M. (2015). A brief argument for, and summary of, the concept of Sonic Virtuality. DanskMusikforskning Online (Special Edition), 81-98. Retrieved from: https://vbn.aau.dk/en/publications/a-brief-argument-for-and-summary-of-the concept-of-sonic-virtuali
Huron, D. (2006). Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA: MIT Press, 2006.
Ihde, D. (2007). Listening and Voice: Phenomenologies of Sound. Albany: State University of New York Press.
Ingold, T. (2007). Against Soundscape. In: Carlyle, A. (Ed.), Autumn Leaves: Sound and the Environment in Artistic Practice (pp. 10-13). Paris: Double Entendre.
Kahn, D. (2006). Sound Art, Art, Music. The Iowa Review Web. Retrieved from: http://iowareview.uiowa.edu
Kapuscinski, J. (2001). Basic Theory of Intermedia Composing with Sounds and Images. (Originally published in 1998 in Monochord: De musica acta, studia et commentarii, Vol. XIX, 43-50. Adam Marszelek Publications, Torun). Retrieved from: http://www.jaroslawkapuscinski.com/pdf/composing-sound-images.pdf
Huang, M., Bridge, H., Kemp, M. J., & Parker, A. J. (2011). Human cortical activity evoked by the assignment of authenticity when viewing works of art. Frontiers in human neuroscience, 5(134). doi:10.3389/fnhum.2011.00134
Hui, A. (2013). The Psychophysical Ear: Musical Experiments, Experimental Sounds, 1840-1910. Cambridge, MA: MIT Press.
Karpf, A. (2007). The Human Voice: The Story of a Remarkable Talent. London: Bloomsbury.
LaBelle, B. (2018). Sonic Agency: Sound and Emergent Forms of Resistance. London: Goldsmiths Press.
Landy, L. (2007). Understanding the art of sound organization. Cambridge, MA: MIT Press.
Lawrence, S. J. D., Formisano, E., Muckli, L., & de Lange, F. P. (2017). Laminar fMRI: Applications for cognitive neuroscience. NeuroImage. Retrieved from: https://doi.org/10.1016/j.neuroimage.2017.07.004
Masumi, B. (2002). Parables for the Virtual Movement, Affect, Sensation. Durham, NC: Duke University Press.
Merleau-Ponty, M. (2002). Phenomenology of Perception (C. Smith, Trans.). London: Routledge.
Mlodinow, L. (2013). Subliminal: How Your Unconscious Mind Rules Your Behavior. New York: Vintage Books.
Moody, N. (2009). Ashitaka: An Audiovisual Instrument (doctoral dissertation). University of Glasgow, UK. Retrieved from: http://theses.gla.ac.uk/1217
Moody, N. (n/d). Ashitaka: An Audiovisual Instrument. Retrieved from: http://www.niallmoody.com/phdstuff/index.htm
Nagai, M. (2011). Listen Compose Listen: A study of perception, process and the spaces between in two works made from listening. Organised Sound, 16(3), 211-219. doi:10.1017/S1355771811000215
Schaeffer, P. (2012). Pierre Schaeffer: In Search of a Concrete Music (C. North & J. Dack, Trans.). Berkeley CA: University of California Press.
Smalley, D. (1996). The listening imagination: Listening in the electroacoustic era. Contemporary Music Review, 13(2), 77-107. doi:10.1080/07494469600640071
Stoever, J. L. (2017). The Sonic Color Line: Race and the Cultural Practices of Listening. New York: New York University Press.
Tiippana, K. (2014). What Is the McGurk Effect? Frontiers in Psychology. Retrieved from: https://doi.org/10.3389/fpsyg.2014.00725
Vincs, K. (2007). Rhizome/myzone: a case study in studio-based dance research. In Barrett, E., & Bolt, B. (Eds.), Practice as research: approaches to creative arts enquiry (pp. 99-112). London: I. B. Tauris.
Voegelin, S. (2010). Listening to Noise and Silence: Towards a Philosophy of Sound Art. London: Bloomsbury.
Voegelin, S. (n/d). Sonic Possible Worlds: In Conversation with Salomé Voegelin. Retrieved from: http://www.fourbythreemagazine.com/issue/world/salom-voegelin-interview
Westerkamp, H. (2015). The Disruptive Nature of Listening. Keynote Address International Symposium on Electronic Art, Vancouver, B.C. Canada. Retrieved from: https://www.sfu.ca/~westerka/writings%20page/articles%20pages/disruptive.html
The journal allow the author(s) to hold the copyright without restrictions. The journal allows the author(s) to retain publishing rights without restrictions.