From the list of sounds required, I decided to first ceate the sound of a “moonjump”. I was confused by this being a required sound, as our game is set on a fictional planet in space, not the moon. I did not want to conform to the conventions of scientific space, where I would assume moonjumops make little to no sound due to the gravity of the moon and how that changes human movement.
I started creating the sound of these “moonjumps” using the sound of me jumping on rocks that we recorded in the foley studio as a group.
I started off by layering the initial “jumping” sound at varying speed and pitches, to create an alien-like sound that sound like it has been affected by a different effect of gravity than on earth. At this point, I decided I wanted to make a sequence of multiple steps, just dropping out some of these layers for some of the steps and manipulated the levels of them for each step, in order to make it sound more realistic that the player is jumping on this unkown planet.
I then moved onto creating the “button sounds”. The button sounds we recorded in the foley studio were very “clicky” in my opinion. Although this is obviously realistic to the sound a button actually makes, I felt it did not fit the theme of the game. I decided to repurpose one of the layers from the moonjump sound I made, and use synthesis to create a more melodic button sound. I then pitched the synth at different pitches, so that each sound could be used for a different button.
My first experience of VR was when I was fifteen, on a school trip to the O2 in Greenwhich. Individiually, one by one, we were given a headset, and made to sit in a chair that could spin 360 degrees, in a small dome-like temporary structure. I watched my classmates, they seemed terrfied. They were screaming, and jumping up off the seat. I was prepared to be frughtened. My turn came, and I was not. The figures looked false and intangible to me, I would liken the experience to going to the Planeterium. I did feel a little ill from the expereince, a feeling similar to motion sickness. I decided on this day that I do not like VR.
Years later, I spoke to someone about this experience, and they too said that in their virtual reality experiences, they too felt motion sickness. I also experience nausea when playing video games, but not when watching long films. I discovered a term called “cybersickness”, which can make people ill due to causing a disruption in bodily function. When the human body is static, but the brain in percieving that it is moving, such as when using virtual reality, confusion is caused in the body, bringing on sickness (in some people).
I thought about how if this cybersickness is a sensation that commonly occurs, how is virtual reality still prevailing? There are VR experiences where the user is able to move physically with the virtual movement, or the user can choose to have a much less immersive experience in terms of the visual.
The game that my group have created is very immersive, but the player must stay in one place to play, using the controls given with the headset to navigate the virtual world. I wonder how cybersickness will affect the experience of our game.
“We are witnessing the shaping up of a world more and more devoted to separation” – Paulo S. H. Favrero.
Favrero’s article is a propsotion of viewing emerging digital words, such as those created in Virtual Reality, through a decolonialist lens. As we are living in an era of post-globalisation, to avoid a growing separation and disconnect from our wider global community, it is imperative to invest in decolonialist thought.
When looking at art, a decolonialst lens is often overlooked, as it is politicised. Drawing on ideas from Sara Ahmed’s On being Included (2012), accepting an institution (institutions, whether that be the gallery or the university, is where art exists and is validated) as built on colonialst structures is to admit to an inherent racism / bigotry, which has brandished the image of what we have come to accept as art. To admit the bigotry would be to unlearn this “brandished image” of art, and therefore everything academia has built in the field of art would have to be deconstructed and rebuilt.
With Virtual Reality being so wrapped up in modernity compared to other art forms, it is easy to apply decolonialist though to the way in which we view our experiences in VR. Favrero proposes that VR is to be looked at through the lens of Buddhist Philosophy. Virtual Reality is largely image based; in Buddhist philosophy, and other religious contexts, images function as “portals”. Favrero is therefore proposing the idea that Visrtual Reality is not just a visual experience that we appreciate for its aesthetic qualities, but rather a world which we immerse ourselves in and can naviagte, completely seperate the one we’d just come from, before placing the headset over our eyes.
In Buddhist thought, seeing invloves the whole body. Virtual Reality is based on a similar concept, where it draws attention to the space between ‘the self’ and ‘the other’- attention is being drawn to the relation between the body and the mind. Not only is this alluding to the aforementioned Buddhist thought, but also the principle of non duality. Virtual reality provides a fragmented map of reality, or the scene that you are being placed in. Fragmentation is what bring Buddhist philosophy and VR close to each other.
To create sounds for the VR collaboration, it was imposrtant to formulate an agreed upon plan between the whole group, which consists of six sound artists and two VR animators. This was to allow a steady and effiecient workflow, where tasks aren’t being left as an afterthought, or other tasks being done multiple times. I would have thought this would have looked like asigning particular tasks to different artists based on their strengnths, i.e. soemone who makes music to make any musical assets, someone who works with field recordings to take charge of foley, etc.
We decided to create a shared google doc for any assets, ideas and contributions towards the project. This allowed for us to collectively identify what needed to be done, and keep track of what had been done to better effectively assign when tasks should be done and by whom. We also identified between us sound artists key stages in creating sound assets, asigning ourselves deadlines for each stage. These stages included pre-production and experimentation, production and manipulating, and mixing in unity.
I would say the stage I felt most comfortable to particpate in was the pre-production and experimentation stage. We worked collaboritvely in the foley and composition studios to collect and create sounds that fit with the soundscape of the theme of our VR experience, which is a game set in space. To be able to use the time in these studios effectively, we created a list of sound assets we aimed to create, and had this list readily accessible to everyone in the shared google drive:
list of sound assets
Using a spreadhseet, the group kept notes on who created which assets, and a link to the asset so they could easily be accessed by anyone in the group. The sounds initially made could then be manipulated by anyone in the group, further adding the process of collaboration to the creation of sounds within the project.
I felt as though there were members of the group who participated more heavily in the initial recording and creation of sound assets, which should allow for others to particpate more heavily in the next stage in creating the sounds: production and manipulating. As I contributed a lot in the foley studio, I decided not to participate in the production stage as much. I did, however, create some ambient music that I thought was remnicsient of space, the theme of the game. It was quite a long piece of music that I thought could be nice as background ambience in one of the scenes, or even be used as a representation of the sound of space.
Although I contributed in the initial creation of sound assets, it would have shown better collaboration skills if I worked on manipulating the sounds that my peers had created, and therefore show my skills in production. The next stage, mixing in unity, requires learning a new skill for all sound artists in the group, therefore, I think it’s best if everyone assists each other by inputting the same level of contribution and effort.
When working with VR, there is the option to create an experience that follows a non-linear structure. This means that the steps the user takes to reach the end of the experience may not be sequential consecutively (i.e. A-B-C), but rather follow a randomised order based on the choices presented to the user (i.e. A-D-F-C).
Creating an experience in Virtual Reality is way less about curating a journey for the user, but rather about providing the ambience and environment for the user to create their own journey. This is why it is important for the artists to create a “spirit of place”. This is referring to the distinctive qulities of the envirnment- the ambience of the envirnment. This term- spirit of place- alludes to an unmanipulated environment, oftentimes with supernatural qualities. Considering the “spirit of place” within art means that art becomes very interwined with spirituality. It begs the questions- is art a spiritual parctice? Must artists hold spiritual beliefs to be able to create art that moves the soul? What about us artists trying to escape the instability of spirituality?
A “sense of place” is much easier to grasp that a “spirit of place”. It is more tangible as it refers to a more “urban” tone. A sense of place is therefore perhaps easier to create than a spirit of place, as it refers to a feeling more constructed by the hands of man. Something within our own capabilities to construct. A sense of place is perhaps a communal sense, as it is manipulated by all the bodies which enter it. It isn’t mystical or “untouchable” the way a spirit of place is.
So when working with Virtual Reality, both these senses are things to be considered, and artists have to carefully think about how they’ll construct them authentically.
Spacialisation a practice by which the composer or audio engineer utilises multi-channel audio systems and mixing techniques to manipulate how sound is percieved in a space. Most commonly used in gallery, art installation settings, and cinematic settings, spacialisation allows for sound to be percieved as “three-dimensional” (Lombardi, 1997), creating a listening expereince much more grounded in realism.
Before delving into playing with spacialisation, it is important to understand the customs and “laws” that surround the practice:
The Pan Laws
Linear Panning: The gains of the left and right channel should sum to 1. The channels will be panned at a 45 degree angle so that the sound reaches a “sweet spot” in the centre. Linear panning commonly has issues with phasing. Constant Power Panning: Similar to linear panning, but the power is proportional to the squared amplitude, boosting the signal at the centre. -4.5dB Pan Law: Considered a compromise between linear and constant panning, the -4.5dB pan law splits the difference between the aforementioned. Ultimately, when a signal is passed centrally, the same output will be in both the left and right channels, which will cause a drop in signal at the centre.
The different surround sound set-ups
Quadrophonics / 4.0 Surround Sound: In this set up, four different speakers are placed in four different corners of a lsitening space, each channel playing back independent signals to one another. Introduced in the 1970s, 4.0 surround sound is designed to allow a listener to feel immersed within the environment of the recording. LCRS: Left, Centre, Right and Surround Channel. Diamond Surround Quad: The same tech spec as Quadrophonics but with the speakers face on to the centre rather than in the corners.
Definitions & Techniques: Recording, Mixing + Playback Mono Sound: Sound Percieved as from a singular source; one dimensional. Stereo Sound: Sound Percieved from multiple directions; allows for playroom with stereo mixing (utilising the left and right channel). Image Position: Ambisonics: A method for recording, mixing and playing back audio in a three dimensional space; audio can be heard from even beneath the listener, as ambisonic labs have mesh floors with loudspeakers placed underneath them.
Institutions + Proponents IRCAM: A research unit based in France, investigating the fields of music and sound. Stockhausen: Non-Profit foundation involved in the advancement of musicology, based on the work of Karlheinz Stockhausen.
The use of spacialisation is particuarly important in VR, as it allows for a more realistic experience. We don’t percieve sound in everyday life the way we do in headohones, or through stero speakers.
“Both virtual reality and the ecological panic are about immersive experiences in which our usual reference point, or illusion of one, has been lost.” – Timothy Morton.
The world of Virtual Reality and creating sound for the purpose of commercialisation may be a daunting one, perhaps even a nauseating one for many. But understanding how sound interacts with the screen, not just in the context of cinema and 2D game, can help further one’s understanding of sound art, and affirm their position as to why they create art.
The term “virtual reality” was coined in the mid 80s by John Lanier, founder of VPL (Virtual Programming Language) research. He used goggles and gloves to create realistic virtual environemnts, which is the basis of how we experience VR today. However, immersive expereinces date back further than that- what we would now recognise as a “4D cinematic experience) was estavblished in 1956, with the world of cinema looking for ways in which the viewer could feel more immersed within the world in which the film / experiene was based in. The history of immersive experiences and imaging doesn’t just reside in the entertainment fields; perhaps the entertainment fields owe a debt to the world of medicine, as the invention of the stereoscope in the 1800s uses very early versions of virtual reality technologies.
For the person partaking in the immersive experience via virtual reality, it is a beautiful experience (unless like myself, VR induces nausea). It allows for them to live within the art, become a part of it. In that sense, VR is a fascillatator of community- the most upheld human value. Perhaps this is why so many people are attracted to the world of VR. It allows them to build connection with not only the art, but the artist whose vision is being displayed. But is this the same notion for the artist fascillitating this imaginitive world? Although a creative proceedure, creating an immersive landscape requires a huge amount of craft, a team of people responsible for producing different assets, and timescales and budgets. With this, the process of creating an immersive experience becomes a job rather than a fun creative project- requring multiple levels of coordination and communication, with some artists on the team having to let go of their values as artists, as the outcome of the work must achieve certain standards that may not adhere to within their personal art. This particuarly applies to the sound designers on the team who are just given lists of assets; the process is much more task based and less collaboritively based than for the visual artists on the team. This therefore places Virtual Reality as a commercialisation of art and, in particular, sound.
In my own VR project, I am on a team made up of six sound artists and two visual artists. Our immersive experience is a game, set in space. We have not assigned set roles so everyone can partake it creating all the different sounds needed; foley, effects, music, etc.
I created this soundscape using synthesis and a heavy use of time stretching. I thought that this fits the aesthetic of “space”. However, it has not beenm used as someone else in the team has taken the role of creating the music.
