Awareness in Motion: heightened awareness through simulated movement in a multimedia environment

Daniël Ploeger
Centre for Research in Opera and Music Theatre, University of Sussex
D.Ploeger@Sussex.ac.uk
 
ABSTRACT
In the 1960s and 1970s, the American artist Bruce Nauman made installations and video works that prompt the observer to explore the awareness of her presence in
relation to the object. The art critic Susan Cross suggests that the effect of these works is related to Michael Fried’s notion of  the “stage presence” of Minimalist works. The works by Nauman that are discussed in this paper are primarily based on the observation of pre-recorded videos or the active participation of the observer in the work. I shall suggest a way to develop Nauman’s strategy for enhancing awareness by confronting the observer with the visual and auditive perception of mobile sounding objects. This will be illustrated by an analysis of my performance installation Box Piece.

Keywords
Multimedia Performance, Moving Sound Sources, Spatial
Music, Installation Art, Post-Minimalism.

1. BRUCE NAUMAN AND THE INFLUENCES OF MINIMALISM
A number of Bruce Nauman’s works from the 1960s and 1970s deal with the awareness of the observer’s perception of the presence of herself and objects around her. Examples of these works are  Lighted Performance Box  (1969), Walk
with Contrapposto (1968) and Performance Corridor (1969). The art critic Susan Cross describes the experience of these works in the context of the “stage presence“  of Minimalist works that Michael Fried suggests in his essay Art and Objecthood [1]. Fried suggests that this stage presence results from the anthropomorphous qualities of Minimalist objects [5]. He gives three reasons for this anthropomorphousness: “First, the size of much [Minimalist] work [...] compares fairly closely with that of the human body. [...] Second, entities or beings encountered in everyday experience in terms that most closely approach the [Minimalist] ideals of the nonrelational, the unitary and the wholistic are other persons. [...] And third, the apparent hollowness of most [Minimalist] work – the quality of having an inside – is almost blatantly anthropomorphic.” [6] The anthropomorphous perception of the object will result in the observer experiencing the relation between object, the space it is in and herself (the subject). Thus the Minimalist work incorporates the subject and the space
around her into the work.
 
Cross suggests that the works by Nauman that I mentioned above evoke a comparable experience by emphasizing the observer’s relationship to the work. Lighted Performance Box consists of a rectangular metal box that is about the
size of a human being and a light that is installed inside the box. The size of the box invites the observer to identify with it (according to Fried), whilst its shape and material are likely to give her a feeling of alienation. The top of the box is open so the light illuminates the space above the object. The light emphasizes the environment of the object and thus makes the observer aware of the object’s relationship to the space around it (which also includes the observer herself).
In Walk with Contrapposto Nauman filmed himself whilst walking through a narrow corridor in a manner referring to the use of posture in classical sculpture. The contrapposto (the torso angled away from the hips) was used to create an illusion of motion. Because Nauman’s head is outside the frame, his image will be perceived as an anonymous person [3]. On one hand the observer encounters a familiar situation (a person walking); on the other hand the situation is uncanny (the person cannot be recognized and the manner of walking is unfamiliar and exagerated). This will lead to a “simultaneous recognition and misrecognition“ [4] of the situation by the observer. The body on the video will become “an abstracted body [...] and a surrogate for the viewer“ [2], but at the same time alienates this body from the observer. Thus Walk with Contrapposto creates a similar experience of the relationship between the work and the observer as Lighted Performance Box does. After Walk with Contrapposto  and other, similar video works such as Bouncing in the Corner, No.1 (1968), Nauman started to further integrate the observer’s presence in his works. In 1969 he exhibited the walls he used to create the corridor in Walk with Contrapposto as Performance Corridor. Instead of watching an anonymous performer on a video performing the action, the observer was now invited to perform the action herself. This is also the case in Video Corridor for San Fransisco (Come Piece) (1969). In this work the observer sees herself on a videoscreen. However, the image has been manipulated (part of the camera is covered and the angle is changed) and the observer is filmed from behind. Therefore the observer will be disoriented when she is first confronted with her image on the video screen. She recognizes that it is her on the video and that the movements of the image on the screen are related to her own movements, but she does not immediately understand the relation between the two. This will draw the observer’s attention to focus on her presence in relation to her surroundings. In other words: this is also an experience of “simultaneous recognition and misrecognition“  that makes the observer realize her presence in relation to the object, the space around it and herself.
 
2. DEVELOPMENT
2.1 Mobile Observer vs. Mobile Object
By exhibiting the walls used for Walk with Contrapposto as Performance Corridor, Nauman moved his focus from letting the observer relate herself to an anonymous
performer on a video to the active participation of the subject; the mobility of the observer had now become a necessity to experience the work. I suggest that an equally interesting possibility lies in a development in a different direction: instead of a mobile observer, one could take the use of mobile objects as a basis for the enhanced experience of presence in the work. Rather than letting the observer perform the actions previously executed by the anonymous performer, one could subsititute the anonymous performer on the video by an anthropomorphously perceived moving object in the observer’s presence and evoke the experience of
“simultaneous recognition and misrecognition“ by confusing the observers sensory perception of the behaviour of this object in space. One could achieve this by providing the observer with contradicting information on the object’s movements. This would be possible when the mentioned anthropomorphous object emits sound.
When an object’s movements can be perceived both visually and auditively, it would be possible to manipulate one of these two sensory inputs in order to create the desired contradicting information on the object’s movement. This strategy will result in an experience related to the discrepancy between the performed movements and the observed movements on the videoscreen that the observer experiences in Video
Corridor.

2.2 Perception of Sounding Objects
It is possible to register information on the movements of an object visually and auditively. The auditive perception of the position of a sounding object is mainly based on the perception of interaural differences in timing and amplitude of the sound, and differences in the perceived sound spectrum caused by spectrum transformations in the outer ear [11]. However, in the case of a moving sound
source, information derived from monaural perception becomes more important [10]. When a sounding object is moving, the observer will register changes in the amplitude of the sound, changes in the perceived frequency of the signal caused by the Doppler effect, a change in the timbre of the sound caused by reflection of the sound on the (static) objects around it and, if there is an additional stationary sound source emitting the same sound, phasing. It is possible to  electronically simulate
these acoustic changes. Thus it is possible to create auditive information on movement that does not correspond with the visually perceived movements. This
strategy can be used to generate the contradicting information on the object’s movement that I mentioned above.

3. SOUNDING OBJECTS IN BOX PIECE
3.1 Installation Setup
The performance installation Box Piece  (figure 5) consists of three large cardboard boxes (71 x 71 x 71 cm) that are attached to the ceiling with ropes. This enables the
performer to move the boxes like pendulums. The boxes are held together with brown tape. Inside boxes 2 and 3, loudspeakers are installed. The performer, who has a microphone taped to his body, takes position in between box 1 and 2 whilst the observers stand in between box 2 and 3. The sound material of the work consists of electronic transformations of the sound that produced by the performer covering his body with brown tape. The performer moves box 2, which will cause a change of the perception of the emitted sound as I mentioned above. The sound emitted by the
loudspeaker in box 3 causes that there will be phasing with the sound from the loudspeaker in box 2 as soon there is movement. Generally the sound of the tape is transformed into pitched sounds that are composed of only a limited number of sine waves. This makes the sound changes that are caused by the movement of box 2 easily perceptible.

3.2 Anthropomorphous Objects
As I showed above, anthropomorphous perception is important in Cross’s experience of “simultaneous recognition and misrecognition“ . The anthropomorphous perception of the boxes in Box Piece is evoked in three different ways:
1. In correspondence with Fried’s analysis of the anthropomorphousness of Minimalist works, the relatively large size of the boxes in MADE facilitates an anthropomorphous perception.
2. Fried also mentions the “apparent hollowness of most [Minimalist] work – the quality of having an inside.“ [7]. When the sound of the brown tape is played back over the loudspeakers inside box 2 and 3, the  reverberation of the sound inside the boxes emphasizes the hollowness that is already suggested by their visual appearance.
3. Experiments by the psychologists Heider and Simmel [8] suggest that when test persons are confronted by a film featuring abstract moving images “the relations among moving shapes elicit specific attributions of personality.” [9] In the case of a large moving object present in the immediate vicinity of the observer, one would
correspondingly expect an attribution of personality to the moving object based on the experienced relations between the observer and the object.

3.3 Simulating Movement
The sound material of Box Piece is processed with a programme written in the programming language Max/MSP. The sound of the performer’s handling of the tape is registered by the microphone attached to the performer’s body. The registered sound is played back through the loudspeakers inside box 1 and 2, and gradually transformed into pitched sounds with a simple harmonic spectrum by means of amplitude modulation. As I mentioned above, the movements of box 2 will result in a change in the perception of the sound it emits. This change will mainly concern phasing with the sound emitted by the loudspeaker in box 3, a change in overall amplitude and, when the movement is fast, a change in pitch caused by the Doppler effect. In order to create auditive information on movement that does not correspond with the visually perceived movements, it would be necessary to simulate these effects electronically. In addition, electronic simulation enables the generation of auditive information that is related to the movements of the actual object to an extent that would be impossible in practice (the movement of the object at a very high speed for example). In order to obtain the necessary information on the characteristics of the sounding object, box 2 was put in an experimental setup where measurements were made of the amplitude change, pitch change and phasing, resulting from different initial impulses given by the performer. This information was then put into a Max/MSP patch that generates data for the modification of the sound material mentioned in the beginning of this paragraph.

3.4 The Performance
In the beginning of the performance, the naked performer starts to wrap up his body with the brown tape. After the wrapping up is completed and the sound is transformed into pitch material, the performer starts moving box 2. Initially, the pitched sounds emitted by the loudspeakers in box 2 and 3 are not manipulated. Thus the observers visually and auditively perceive the actual movements of box 2. The observers cannot perceive the exact nature of the performer’s action since their vision is partly blocked by the box. This secures the continued perception of the box’s movement as (partly) autonomous (and thus as anthropomorphous). A situation is created where it is not really clear whether it is the box’s movements that dictate the performer’s movements, or the performer that controls the movements of the box.
Gradually, the emitted sound is electronically manipulated in such a way that the auditive cues of the box's movement are simulated. Towards the end of the performance, box 2 is given an initial impulse and then left moving freely. The amplitude of this pendulum movement will be damped naturally, which will also cause the naturally occurring changes in sound material to decrease. Simultaneously, however, the intensity and quantity of simulated sound transformations increase gradually. In effect this means that the observer sees the movement of the object decreasing, whereas she hears an overall increase of the object’s movement.

4. SUMMARY
In this paper, I suggested that it is possible to develop Bruce Nauman’s strategy for enhancing awareness, by confronting the observer with the visual and auditive perception of mobile sounding objects. Nauman's use of contradicting visual information on the position of the observer's body or objects surrounding it, could be replaced by a juxtaposition of contradicting aural and visual information on an object's location in relation to the observer. In my performance installation “Box Piece”, I used this hypothesis as one of the theoretical starting points for the conception of the work. A moving sounding object in combination with computer generated aural motion cues was used to enable the occurrence of contradicting visual and aural motion cues.

5. REFERENCES
[1] Cross, Susan, ’Bruce Nauman: Theaters of Experience’ in Bruce Nauman: Theaters of Experience (New York: D.A.P., 2003), p. 13.
[2] Ibid. p. 14.
[3] Ibid. p. 15.
[4] Ibid. p. 17.
[5] Fried, Michael, “Art and Objecthood”, Art in theory 1900-1990 : an anthology of changing ideas, edited by Charles Harrison and Paul Wood (London: Blackwell, 1992), p. 826.
[6] Ibid. p. 827.
[7] Ibid. p. 827.
[8] Heider, F. and M. Simmel, ’An experimental study of apparent behavior’, American Journal of Psychology, no. 57 (1944), pp. 243-259.
[9] Ibid. p. 96.
[10] Perrott, David R. and Thomas Z. Strybel, „Some Observations Regarding Motion Without Direction“, Binaural and Spatial Hearing in Real and Virtual Environments, ed. by Robert G. Gilkey and Timothy R. Anderson (Mahwah NJ: 1997), pp. 289-290.
[11] Wightman, Frederic L. and Doris J. Kistler, „Factors Affecting the Relative Salience of Sound Localization Cues“ Binaural and Spatial Hearing in Real and Virtual Environments, ed. by Robert G. Gilkey and Timothy R. Anderson (Mahwah NJ: 1997), p.1.