GENERATION KILL for electric guitar, percussion, violin, cello, 4 performers with gamecontrollers, 4 live videoâ€™s & live-electronics
Commissioned by SWR for the Donaueschinger Musiktage 2012 Dedicated to and premiered by my dear friends of the Nadar Ensemble
Stefan Prins (2012)
Generation Kill Stefan Prins (2012)
I. OPERATING THE GAMECONTROLLER A
1. GENERAL FUNCTIONING: ROTATIONS, AUDIO & VIDEO SECTION, TRIGGERING
2. SELECTING AUDIO- & VIDEO-BUFFERS
Only with the Sony Playstation3 gamecontroller will the software work properly. This gamecontroller has several accelerometers built-in, which detect rotations along several axis. Furthermore, it has two X-Y joysticks (J.St.-L & J.St.-R), which can also be pressed as a button. In the middle of the controller, you find three buttons: “select”, “start” and the “PS3”-button. All other buttons on the gamecontroller are pressure sensitive.
Four (plus one) audio/video-buffers are preloaded on startup of the application. The performer always has to choose which one of the four buffers to use for the playback.Toggling between these different buffers can be done easily during performance using the thumb-pads: the left thumb pad (Pad-L) and the right thumb pad (Pad-R). Pad-L selects which video-buffer is selected, while Pad-R selects which audio-buffer is used. In both cases, buffer 1 is selected using the upper button, the button to the right selects buffer 2, the lower button selects buffer 3, the left button selects buffer 4. Note: with the presets are activated (see below) the correct buffers are automatically loaded.
As shown in Fig.2, the gamecontroller is divided in two sections (not considering the three buttons in the middle of the controller: “select”, “start” and “playstation” button): • LEFT half/section: this half is used for manipulations related to video. • RIGHT half/section: this half is used for manipulations related to audio. The L1, L2 and R1 buttons are the “triggers” (or “shoot”) buttons and activate either the video images (L1, L2) or the audio of these images (R1). Audio and video can be activated independently, by pressing either L1/L2 (only video) or R1 (only audio).
3. CHOOSING AND PLAYING THE LIVE & PRERECORDED AUDIO / VIDEO The L1 and R1 buttons activate respectively the selected (with the thumb-pad) prerecorded videobuffers and/or audio-buffer. To activate the live-camera (webcam) of the instrumentalist, the performer presses the L2 button, together with one of the buttons of Pad-L. When pressing the upper button of the left thumb-pad while the L2 button is pressed, the webcam of the instrumentalist is activated. When pressing the left button of Pad-L while the L2 button is pressed, the webcam of the gamecontroller-performer
is activated. When pressing the right and/or lower button of Pad-L while the L2 button is pressed, the webcam of instrumentalist X/Y is activated. 4. PLAYBACK SPEED OF AUDIO / VIDEO As shown in Fig.1, the gamecontroller has two axis around which it rotates: When one rotates around the A Axis, the left point of the playback-loop moves across the buffer (see also fig.4). When rotating the gamecontroller to the left (as seen from the player’s perspective), the cursor will move to the left in the buffer & vice versa. When one rotates around the B axis, the playback speed of the loop is affected. Rotating the “head” of the gamecontroller towards the floor slows down the playback speed, rotating it towards the sky speeds up the playback speed. 5. RECORDING & PLAYING BACK THE (INSTRUMENTALIST’S) WEBCAM (AND THE AFFILIATED AUDIO) Both the left and right joysticks have also a button-function: they can be pushed down. By pushing the right joystick button (J.St.-R) you start recording the input of the first webcam (pointed at the instrumentalist). By pushing it again, you stop the recording. When recording the Idle button in the capture module will turn red and will say recording (see fig.4). Each time a recording is being made, it is concatenated to the previous recording. At specific points in the piece (with a specific preset) this recording is then loaded and will replace one of the other four buffers. It can then be used for playback and manipulation as with all the other buffers.
9. FORWARDS & BACKWARDS PLAYBACK When pushing down the left joystick button, the playback direction changes from forwards to backwards and vice versa. 10. BLACK SCREEN-MODE & FREEZE SCREEN MODE When pushing down the left joystick-button together with the select button (first select, then joystick), you toggle between black screen mode and freeze-screen mode. 11. RESTART & CONTINUE MODE There are two modes for playback: the so-called “restart” mode (in which every time L1/R1 is pushed again, the buffer is read from the beginning of the selected portion) and the so-called “continue” mode (in which every time L1/R1 is pushed, the selected portion of the buffer is being read starting from the position where it stopped just before) 12. VIDEO DISTORTIONS The video-“distortion” can be activated by pushing the R2 button. The “depth” of the distortion of the image is relative to how hard the R2 button is pressed.
6. SELECTING THE ACTIVE PART OF AN AUDIO/VIDEO LOOP As you can see in fig.4, there are two waveform viewers for the audio. The upper waveform viewer shows the full preloaded audio buffer. The lower waveform viewer shows the selected chunk (loop) of this buffer. To move the left (starting) point of the loop to another position in the buffer, rotate the gamecontroller along the A-axis (as if stearing a car). To move the right (ending) point of the buffer, use the left joystick (J.St.-L) and move to the left (to make the loop shorter) or to the right (to make the loop longer). 7. ZOOMING IN/OUT ON A SELECTION OF THE BUFFER It’s also possible to zoom in/out on the selected part of the buffer, by pulling the right joystick (J.St.-R) upwards once. On the upper waveform viewer, you will then see on which part of the buffer you have zoomed in, while on the lower waveform viewer you’ll see the zoomed buffer itself. It is then possible to move within this (zoomed-in) selection as described in 6. It’s also possible to zoom-in even more on the part you select in the lower buffer. This can be done untill the zoomed-in part of the buffer is one sample big. To zoom out all the way again to the whole buffer, pull down the right joystick once. 8. STORING AND RECALLING BUFFER POSITIONS (WITH ITS RELATED LOOP SIZE) By pulling the right joystick (J.St.-R) to the right once, you store the current buffer position with its related loop size. This is stored until you pull the joystick to the right again to store another preset. When pulling the right joystick once to the left, this preset is recalled.
II. THE USER INTERFACE1 Each player uses the corresponding standalone application (“GenKill-player1” for player 1 etc.). Player 1 is the partner of the cellist, player 2 of the percussionist, player 3 of the electric guitarist, player 4 of the violinist. Before opening the program, make sure that the soundcard is connected to the computer and turned on, and that the gamecontroller and 2 webcams are connected to the computer through USB-cables (you might need to buy a usb-hub (preferably with external power supply) to make sure you have enough usb-slots available) After double-clicking the program will open automatically. You’ll see three windows: the “MAX”window , which reports error messages (you might get several error-messages on start-up, but you can ignore those), the “GenKill”-window, which is the control room (see image below), and another window which has a generic name (like u877008625), which is the projected video-image . During performance, you’ll have to watch only the “GenKill” window (you can close the “MAX” window). The “GenKill” window consists of 10 modules, each one with a specific function. Below some explanation about the modules. Not every meter/button is explained, as some are self-explanatory and others are not important for the performer. 1. Playback module In the window on the left you can see the projected video-image, in case it comes from one of the preloaded buffers. The two bigger rectangular boxes V: and A: tell you which video respectively audio-buffer is being used. On the waveform viewers to the right of this module, you can see both the full (selected) buffer (on top), and the active (zoomed-in) part of this buffer (below). In between these two waveform viewers, there is a speed-indicator (middle = “normal” speed), and three other buttons which change states when you press them: - Continue mode/Restart: when in continue mode, the selected buffer is, when triggered, being read out starting where it stopped before; in restart mode, the buffer is always being read out from the beginning when it is triggered again - Forwards/Backwards: the buffer is being read from left to right (forwards) or vice versa (backwards) - Blackscreen/Freeze: when the buffer isn’t triggered anymore, the screen will go to black when in blackscreen mode; when in freezescreen mode, the image on the screen will just freeze in the last frame it had. 2. Output module
On the screen in this module you can see the image which is being projected on the screen (it’s a double of the other window). DSP button: hit this button to open the DSP-window in which you can select the soundcard (audio driver) you’re using. The little icon of a speaker on the lower right corner of this module should be turned on (blue) before the audio-processing is activated.Normally should be done automatically when the “INITIALIZATION” cue is triggered in the cuescript module (see below), but it can also be done manually. 3. Input module This module deals with the two webcams. On the left, you’ll see the webcam filming the instrumentalist, on the right the webcam filming yourself. Push getdevices for the module to detect the active cameras. Then push on the open/close button, to open the “channel” (and be sure that the ACTIVE toggle is checked on the upper right corner). 4. ps3 module This module visualizes the actions performed on the gamecontroller. In case moving the gamecontroller has no effect on the meters/buttons in this module, the program might be “looking” at the wrong device. Therefore it’s important to click getdevices, then select PLAYSTATION(R)3 Controller (or sometimes PLAYSTATION(R)3 Controller 2) from the dropdown menu on top of the module. Once this is done, click the Start Polling button just below the menu, to make sure the program “listens” to the gamecontroller. 5. Capture module This module monitors the live-recording of the input of the instrumentalist’s webcam. The recorded video-files are automatically stored in the correct folder inside the program – no need for interacting with this module! 6. Mixer module This module, which doesn’t need to be manipulated on screen, routes the audio and video-inputs to “channels”. Channel 4 = the prerecorded buffers (as shown in the Playback module) Channel 5 = the first webcam Channel 6 = the second webcam Channel 7 = the webcam of one of the other instruments (these are being sent to the program through the network to whcih all computers are connected) Channel 8 = the webcam of yet another of the instruments (these are being sent to the program through the network to whcih all computers are connected) 7. Treatment module This module, which doesn’t need to be manipulated on screen, monitors the amount of audio or video treatment.
The programming of the GenKill-application was done by Josiah Oberholtzer (2012) for the composition Generation Kill
8. Config module This module configurates the network to which all computers are connected. Push DISCOVER once you’re in a network with other computers. 9. Networking module In this module, you see which IP-address is used by which performer, into which network you’re operating and as which player. These settings are automatically loaded, so you don’t need to interact with this module. 10. Cuescript module This module contains all the cues (presets) of the piece. In the dropdown menu you can select the correct preset. After starting up the program, choose the menu-item “INITIALIZATION”, which initializes the patch and loads all necessary presets to make the program performance-ready. After this is done, choose the menu-item which corresponds to the performer you are. If you’re i.e. the partner of the cellist, you choose “VIRTUAL CELLO SETUP”. Then all cello video-samples are loaded into the buffers (which might take a while) and made ready for performance. REMARK: after having done this, It might be necessary again to push “START POLLING” in the ps3 module!
Fig. 4: GenKill window,the “control room”
III. CHECKLIST TO GET THE APPLICATION RUNNING SYSTEM REQUIREMENTS: This application has been written in MAX5.1.9, makes use of Jamoma 0.5.6 beta7 and works perfectly on a MacBook Pro 2 GHz intel Core 2 Duo (4 GB 1067 MHz DDR3 RAM & a NVIDIA GeForce 9400M graphics card) running on Mac OSX 10.6.8. 1. connect the computer to the internet router with an ehternet cable & make sure the wireless internet connection is turned off 2. connect soundcard to computer and turn on soundcard 3. connect gamecontroller to computer through USB. 4. connect the webcams to computer through USB (if necessary, use a USB-hub –with external power supply- to connect all USB-devices to the computer) 5. double-click the GenKill-player1 (or player2, 3 or 4) application to start the application 6. do the following in the GenKill application: a. in the output module: click on DSP and set the audio driver to the soundcard you have connected to your computer previously b. press on the ps3 button in the middle of the gamecontroller, to activate the gamecontroller c. in the ps3 module: i. click on the Get Devices button ii. select PLAYSTATION(R)3 Controller from the dropdown menu on top of the module iii. click the Start Polling button iv. now any movement or button being pushed on the gamecontroller should be reflected in this module d. in the input module: i. click on the Get Devices button ii. click on the ACTIVE toggle iii. select the correct webcam on the dropdown menu’s iv. click on the open button v. now you should see the video as it is captured by the webcams e. in the cuescript module: i. select INITIALIZATION in the drop-down menu ii. select VIRTUAL CELLO SETUP / VIRTUAL PERCUSSION SETUP / VIRTUAL E-GTR SETUP / VIRTUAL VIOLIN SETUP according to your part (NOTE: it might take a minute before all video’s are loaded into RAM at this point) iii. it might be possible that you need to go back to step 5.c. at this pointm to activate the gamecontroller again. f. In the output module: be sure that the button with the speaker-icon is on (blue). If it’s not, click on it.
More info: firstname.lastname@example.org // www.stefanprins.be